Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-8912

[fiware-stackoverflow] adding Apache flume elastic search sink

    Details

      Description

      Created question in FIWARE Q/A platform on 08-03-2017 at 17:03
      Please, ANSWER this question AT https://stackoverflow.com/questions/42676814/adding-apache-flume-elastic-search-sink

      Question:
      adding Apache flume elastic search sink

      Description:
      I want to use Apache flume Elastic search native sink in cygnus.
      How can I configure cygnus to use this native sink?

      Do I put the native jar file in Apache flume lib and configure the agent as the folowing:

      1. Name the components on this agent
        a1.sources = r1
        a1.sinks = k1
        a1.channels = c1
      1. Describe/configure the source
        a1.sources.r1.type = netcat
        a1.sources.r1.bind = localhost
        a1.sources.r1.port = 44444
      1. Describe the sink
        a1.sinks.k1.type = logger
      1. Use a channel which buffers events in memory
        a1.channels.c1.type = memory
        a1.channels.c1.capacity = 1000
        a1.channels.c1.transactionCapacity = 100
      1. Bind the source and sink to the channel
        a1.sources.r1.channels = c1
        a1.sinks.k1.channel = c1

      and then I start two cygnus instance one with for cygnus-ngsi agent and the other for elastic search native sink?
      Can't find how to run a cygnus instance (I installed cygnus from sources so I can't use linux services to start it)

      And is it possible to use the same source for cygnus-ngsi and this new sink? cause I need the same source for both.

      Thanks and best regards.

        Activity

        backlogmanager Backlog Manager created issue -
        Hide
        backlogmanager Backlog Manager added a comment -

        2017-05-22 15:11|CREATED monitor | # answers= 1, accepted answer= True

        Show
        backlogmanager Backlog Manager added a comment - 2017-05-22 15:11|CREATED monitor | # answers= 1, accepted answer= True
        backlogmanager Backlog Manager made changes -
        Field Original Value New Value
        Component/s FIWARE-TECH-HELP [ 10278 ]
        backlogmanager Backlog Manager made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Hide
        backlogmanager Backlog Manager added a comment -

        2017-05-22 18:08|UPDATED status: transition Answer| # answers= 1, accepted answer= True

        Show
        backlogmanager Backlog Manager added a comment - 2017-05-22 18:08|UPDATED status: transition Answer| # answers= 1, accepted answer= True
        backlogmanager Backlog Manager made changes -
        Resolution Done [ 10000 ]
        Status In Progress [ 3 ] Closed [ 6 ]
        Hide
        backlogmanager Backlog Manager added a comment -

        2017-05-22 21:08|UPDATED status: transition Finish| # answers= 1, accepted answer= True

        Show
        backlogmanager Backlog Manager added a comment - 2017-05-22 21:08|UPDATED status: transition Finish| # answers= 1, accepted answer= True
        fla Fernando Lopez made changes -
        HD-Enabler Cygnus [ 11304 ]
        Description
        Created question in FIWARE Q/A platform on 08-03-2017 at 17:03
        {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/42676814/adding-apache-flume-elastic-search-sink


        +Question:+
        adding Apache flume elastic search sink

        +Description:+
        I want to use Apache flume Elastic search native sink in cygnus.
        How can I configure cygnus to use this native sink?

        Do I put the native jar file in Apache flume lib and configure the agent as the folowing:

        # Name the components on this agent
        a1.sources = r1
        a1.sinks = k1
        a1.channels = c1

        # Describe/configure the source
        a1.sources.r1.type = netcat
        a1.sources.r1.bind = localhost
        a1.sources.r1.port = 44444

        # Describe the sink
        a1.sinks.k1.type = logger

        # Use a channel which buffers events in memory
        a1.channels.c1.type = memory
        a1.channels.c1.capacity = 1000
        a1.channels.c1.transactionCapacity = 100

        # Bind the source and sink to the channel
        a1.sources.r1.channels = c1
        a1.sinks.k1.channel = c1


        and then I start two cygnus instance one with for cygnus-ngsi agent and the other for elastic search native sink?
        Can't find how to run a cygnus instance (I installed cygnus from sources so I can't use linux services to start it)

        And is it possible to use the same source for cygnus-ngsi and this new sink? cause I need the same source for both.

        Thanks and best regards.
        Created question in FIWARE Q/A platform on 08-03-2017 at 17:03
        {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/42676814/adding-apache-flume-elastic-search-sink


        +Question:+
        adding Apache flume elastic search sink

        +Description:+
        I want to use Apache flume Elastic search native sink in cygnus.
        How can I configure cygnus to use this native sink?

        Do I put the native jar file in Apache flume lib and configure the agent as the folowing:

        # Name the components on this agent
        a1.sources = r1
        a1.sinks = k1
        a1.channels = c1

        # Describe/configure the source
        a1.sources.r1.type = netcat
        a1.sources.r1.bind = localhost
        a1.sources.r1.port = 44444

        # Describe the sink
        a1.sinks.k1.type = logger

        # Use a channel which buffers events in memory
        a1.channels.c1.type = memory
        a1.channels.c1.capacity = 1000
        a1.channels.c1.transactionCapacity = 100

        # Bind the source and sink to the channel
        a1.sources.r1.channels = c1
        a1.sinks.k1.channel = c1


        and then I start two cygnus instance one with for cygnus-ngsi agent and the other for elastic search native sink?
        Can't find how to run a cygnus instance (I installed cygnus from sources so I can't use linux services to start it)

        And is it possible to use the same source for cygnus-ngsi and this new sink? cause I need the same source for both.

        Thanks and best regards.
        fla Fernando Lopez made changes -
        Assignee Backlog Manager [ backlogmanager ]
        fla Fernando Lopez made changes -
        Fix Version/s 2021 [ 12600 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        2h 56m 1 Backlog Manager 22/May/17 6:07 PM
        In Progress In Progress Closed Closed
        3h 1 Backlog Manager 22/May/17 9:07 PM

          People

          • Assignee:
            backlogmanager Backlog Manager
            Reporter:
            backlogmanager Backlog Manager
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: