Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-8367

[fiware-stackoverflow] adding Apache flume elastic search sink

        Details

          Description

          Created question in FIWARE Q/A platform on 08-03-2017 at 17:03
          Please, ANSWER this question AT http://stackoverflow.com/questions/42676814/adding-apache-flume-elastic-search-sink

          Question:
          adding Apache flume elastic search sink

          Description:
          I want to use Apache flume Elastic search native sink in cygnus.
          How can I configure cygnus to use this native sink?

          Do I put the native jar file in Apache flume lib and configure the agent as the folowing:

          1. Name the components on this agent
            a1.sources = r1
            a1.sinks = k1
            a1.channels = c1
          1. Describe/configure the source
            a1.sources.r1.type = netcat
            a1.sources.r1.bind = localhost
            a1.sources.r1.port = 44444
          1. Describe the sink
            a1.sinks.k1.type = logger
          1. Use a channel which buffers events in memory
            a1.channels.c1.type = memory
            a1.channels.c1.capacity = 1000
            a1.channels.c1.transactionCapacity = 100
          1. Bind the source and sink to the channel
            a1.sources.r1.channels = c1
            a1.sinks.k1.channel = c1

          and then I start two cygnus instance one with for cygnus-ngsi agent and the other for elastic search native sink?
          Can't find how to run a cygnus instance (I installed cygnus from sources so I can't use linux services to start it)

          And is it possible to use the same source for cygnus-ngsi and this new sink? cause I need the same source for both.

          Thanks and best regards.

            Activity

            Descending order - Click to sort in ascending order
            Transition Time In Source Status Execution Times Last Executer Last Execution Date
            Open Open In Progress In Progress
            		17h 59m 1 Backlog Manager 09/Mar/17 12:04 PM
            In Progress In Progress Answered Answered
            		2h 59m 1 Backlog Manager 09/Mar/17 3:04 PM
            Answered Answered Closed Closed
            		1d 3h 1 Backlog Manager 10/Mar/17 6:04 PM
            fla Fernando Lopez made changes -
            Fix Version/s 2021 [ 12600 ]
            Hide
            Permalink
            backlogmanager Backlog Manager added a comment -

            2017-03-10 18:05|UPDATED status: transition Finish| # answers= 1, accepted answer= True

            Show
            backlogmanager Backlog Manager added a comment - 2017-03-10 18:05|UPDATED status: transition Finish| # answers= 1, accepted answer= True
            backlogmanager Backlog Manager made changes -
            Resolution Done [ 10000 ]
            Status Answered [ 10104 ] Closed [ 6 ]
            Hide
            Permalink
            backlogmanager Backlog Manager added a comment -

            2017-03-09 15:05|UPDATED status: transition Answered| # answers= 1, accepted answer= False

            Show
            backlogmanager Backlog Manager added a comment - 2017-03-09 15:05|UPDATED status: transition Answered| # answers= 1, accepted answer= False
            backlogmanager Backlog Manager made changes -
            Status In Progress [ 3 ] Answered [ 10104 ]
            Hide
            Permalink
            backlogmanager Backlog Manager added a comment -

            2017-03-09 12:05|UPDATED status: transition Answer| # answers= 1, accepted answer= False

            Show
            backlogmanager Backlog Manager added a comment - 2017-03-09 12:05|UPDATED status: transition Answer| # answers= 1, accepted answer= False
            backlogmanager Backlog Manager made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            fla Fernando Lopez made changes -
            HD-Enabler Cosmos [ 10872 ]
            Description
            Created question in FIWARE Q/A platform on 08-03-2017 at 17:03
            {color: red}Please, ANSWER this question AT{color} http://stackoverflow.com/questions/42676814/adding-apache-flume-elastic-search-sink


            +Question:+
            adding Apache flume elastic search sink

            +Description:+
            I want to use Apache flume Elastic search native sink in cygnus.
            How can I configure cygnus to use this native sink?

            Do I put the native jar file in Apache flume lib and configure the agent as the folowing:

            # Name the components on this agent
            a1.sources = r1
            a1.sinks = k1
            a1.channels = c1

            # Describe/configure the source
            a1.sources.r1.type = netcat
            a1.sources.r1.bind = localhost
            a1.sources.r1.port = 44444

            # Describe the sink
            a1.sinks.k1.type = logger

            # Use a channel which buffers events in memory
            a1.channels.c1.type = memory
            a1.channels.c1.capacity = 1000
            a1.channels.c1.transactionCapacity = 100

            # Bind the source and sink to the channel
            a1.sources.r1.channels = c1
            a1.sinks.k1.channel = c1


            and then I start two cygnus instance one with for cygnus-ngsi agent and the other for elastic search native sink?
            Can't find how to run a cygnus instance (I installed cygnus from sources so I can't use linux services to start it)

            And is it possible to use the same source for cygnus-ngsi and this new sink? cause I need the same source for both.

            Thanks and best regards.
            		 Created question in FIWARE Q/A platform on 08-03-2017 at 17:03
            {color: red}Please, ANSWER this question AT{color} http://stackoverflow.com/questions/42676814/adding-apache-flume-elastic-search-sink


            +Question:+
            adding Apache flume elastic search sink

            +Description:+
            I want to use Apache flume Elastic search native sink in cygnus.
            How can I configure cygnus to use this native sink?

            Do I put the native jar file in Apache flume lib and configure the agent as the folowing:

            # Name the components on this agent
            a1.sources = r1
            a1.sinks = k1
            a1.channels = c1

            # Describe/configure the source
            a1.sources.r1.type = netcat
            a1.sources.r1.bind = localhost
            a1.sources.r1.port = 44444

            # Describe the sink
            a1.sinks.k1.type = logger

            # Use a channel which buffers events in memory
            a1.channels.c1.type = memory
            a1.channels.c1.capacity = 1000
            a1.channels.c1.transactionCapacity = 100

            # Bind the source and sink to the channel
            a1.sources.r1.channels = c1
            a1.sinks.k1.channel = c1


            and then I start two cygnus instance one with for cygnus-ngsi agent and the other for elastic search native sink?
            Can't find how to run a cygnus instance (I installed cygnus from sources so I can't use linux services to start it)

            And is it possible to use the same source for cygnus-ngsi and this new sink? cause I need the same source for both.

            Thanks and best regards.
            HD-Chapter Data [ 10838 ]
            fla Fernando Lopez made changes -
            Assignee Francisco Romero [ frb ]
            backlogmanager Backlog Manager made changes -
            Field Original Value New Value
            Component/s FIWARE-TECH-HELP [ 10278 ]
            Hide
            Permalink
            backlogmanager Backlog Manager added a comment -

            2017-03-08 18:05|CREATED monitor | # answers= 0, accepted answer= False

            Show
            backlogmanager Backlog Manager added a comment - 2017-03-08 18:05|CREATED monitor | # answers= 0, accepted answer= False
            backlogmanager Backlog Manager created issue -

              People

              • Assignee:
                frb Francisco Romero
                Reporter:
                backlogmanager Backlog Manager
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: