Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-8367

[fiware-stackoverflow] adding Apache flume elastic search sink

    Details

      Description

      Created question in FIWARE Q/A platform on 08-03-2017 at 17:03
      Please, ANSWER this question AT http://stackoverflow.com/questions/42676814/adding-apache-flume-elastic-search-sink

      Question:
      adding Apache flume elastic search sink

      Description:
      I want to use Apache flume Elastic search native sink in cygnus.
      How can I configure cygnus to use this native sink?

      Do I put the native jar file in Apache flume lib and configure the agent as the folowing:

      1. Name the components on this agent
        a1.sources = r1
        a1.sinks = k1
        a1.channels = c1
      1. Describe/configure the source
        a1.sources.r1.type = netcat
        a1.sources.r1.bind = localhost
        a1.sources.r1.port = 44444
      1. Describe the sink
        a1.sinks.k1.type = logger
      1. Use a channel which buffers events in memory
        a1.channels.c1.type = memory
        a1.channels.c1.capacity = 1000
        a1.channels.c1.transactionCapacity = 100
      1. Bind the source and sink to the channel
        a1.sources.r1.channels = c1
        a1.sinks.k1.channel = c1

      and then I start two cygnus instance one with for cygnus-ngsi agent and the other for elastic search native sink?
      Can't find how to run a cygnus instance (I installed cygnus from sources so I can't use linux services to start it)

      And is it possible to use the same source for cygnus-ngsi and this new sink? cause I need the same source for both.

      Thanks and best regards.

        Activity

        backlogmanager Backlog Manager created issue -
        Hide
        backlogmanager Backlog Manager added a comment -

        2017-03-08 18:05|CREATED monitor | # answers= 0, accepted answer= False

        Show
        backlogmanager Backlog Manager added a comment - 2017-03-08 18:05|CREATED monitor | # answers= 0, accepted answer= False
        backlogmanager Backlog Manager made changes -
        Field Original Value New Value
        Component/s FIWARE-TECH-HELP [ 10278 ]
        fla Fernando Lopez made changes -
        Assignee Francisco Romero [ frb ]
        fla Fernando Lopez made changes -
        HD-Enabler Cosmos [ 10872 ]
        Description
        Created question in FIWARE Q/A platform on 08-03-2017 at 17:03
        {color: red}Please, ANSWER this question AT{color} http://stackoverflow.com/questions/42676814/adding-apache-flume-elastic-search-sink


        +Question:+
        adding Apache flume elastic search sink

        +Description:+
        I want to use Apache flume Elastic search native sink in cygnus.
        How can I configure cygnus to use this native sink?

        Do I put the native jar file in Apache flume lib and configure the agent as the folowing:

        # Name the components on this agent
        a1.sources = r1
        a1.sinks = k1
        a1.channels = c1

        # Describe/configure the source
        a1.sources.r1.type = netcat
        a1.sources.r1.bind = localhost
        a1.sources.r1.port = 44444

        # Describe the sink
        a1.sinks.k1.type = logger

        # Use a channel which buffers events in memory
        a1.channels.c1.type = memory
        a1.channels.c1.capacity = 1000
        a1.channels.c1.transactionCapacity = 100

        # Bind the source and sink to the channel
        a1.sources.r1.channels = c1
        a1.sinks.k1.channel = c1


        and then I start two cygnus instance one with for cygnus-ngsi agent and the other for elastic search native sink?
        Can't find how to run a cygnus instance (I installed cygnus from sources so I can't use linux services to start it)

        And is it possible to use the same source for cygnus-ngsi and this new sink? cause I need the same source for both.

        Thanks and best regards.
        Created question in FIWARE Q/A platform on 08-03-2017 at 17:03
        {color: red}Please, ANSWER this question AT{color} http://stackoverflow.com/questions/42676814/adding-apache-flume-elastic-search-sink


        +Question:+
        adding Apache flume elastic search sink

        +Description:+
        I want to use Apache flume Elastic search native sink in cygnus.
        How can I configure cygnus to use this native sink?

        Do I put the native jar file in Apache flume lib and configure the agent as the folowing:

        # Name the components on this agent
        a1.sources = r1
        a1.sinks = k1
        a1.channels = c1

        # Describe/configure the source
        a1.sources.r1.type = netcat
        a1.sources.r1.bind = localhost
        a1.sources.r1.port = 44444

        # Describe the sink
        a1.sinks.k1.type = logger

        # Use a channel which buffers events in memory
        a1.channels.c1.type = memory
        a1.channels.c1.capacity = 1000
        a1.channels.c1.transactionCapacity = 100

        # Bind the source and sink to the channel
        a1.sources.r1.channels = c1
        a1.sinks.k1.channel = c1


        and then I start two cygnus instance one with for cygnus-ngsi agent and the other for elastic search native sink?
        Can't find how to run a cygnus instance (I installed cygnus from sources so I can't use linux services to start it)

        And is it possible to use the same source for cygnus-ngsi and this new sink? cause I need the same source for both.

        Thanks and best regards.
        HD-Chapter Data [ 10838 ]
        backlogmanager Backlog Manager made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Hide
        backlogmanager Backlog Manager added a comment -

        2017-03-09 12:05|UPDATED status: transition Answer| # answers= 1, accepted answer= False

        Show
        backlogmanager Backlog Manager added a comment - 2017-03-09 12:05|UPDATED status: transition Answer| # answers= 1, accepted answer= False
        backlogmanager Backlog Manager made changes -
        Status In Progress [ 3 ] Answered [ 10104 ]
        Hide
        backlogmanager Backlog Manager added a comment -

        2017-03-09 15:05|UPDATED status: transition Answered| # answers= 1, accepted answer= False

        Show
        backlogmanager Backlog Manager added a comment - 2017-03-09 15:05|UPDATED status: transition Answered| # answers= 1, accepted answer= False
        backlogmanager Backlog Manager made changes -
        Resolution Done [ 10000 ]
        Status Answered [ 10104 ] Closed [ 6 ]
        Hide
        backlogmanager Backlog Manager added a comment -

        2017-03-10 18:05|UPDATED status: transition Finish| # answers= 1, accepted answer= True

        Show
        backlogmanager Backlog Manager added a comment - 2017-03-10 18:05|UPDATED status: transition Finish| # answers= 1, accepted answer= True
        fla Fernando Lopez made changes -
        Fix Version/s 2021 [ 12600 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        17h 59m 1 Backlog Manager 09/Mar/17 12:04 PM
        In Progress In Progress Answered Answered
        2h 59m 1 Backlog Manager 09/Mar/17 3:04 PM
        Answered Answered Closed Closed
        1d 3h 1 Backlog Manager 10/Mar/17 6:04 PM

          People

          • Assignee:
            frb Francisco Romero
            Reporter:
            backlogmanager Backlog Manager
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: