Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-6713

FIWARE.Question.Tech.Data.BigData-Analysis.fiware cygnus channels are full

    Details

      Description

      Created question in FIWARE Q/A platform on 02-06-2016 at 10:06
      Please, ANSWER this question AT http://stackoverflow.com/questions/37586890/fiware-cygnus-channels-are-full

      Question:
      fiware cygnus channels are full

      Description:
      I’ve checked the Cygnus logs in the morning and see these channel errors.
      It seems that ckan-channel is full and can’t continue the transactions.

      What do you suggest us for this error?

      16/06/01 08:11:58 WARN http.HTTPSource: Error appending event to channel. Channel might be full. Consider increasing the channel capacity or make sure the sinks perform faster.
      org.apache.flume.ChannelException: Unable to put batch on required channel: org.apache.flume.channel.MemoryChannel

      {name: ckan-channel}

      at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:200)
      at org.apache.flume.source.http.HTTPSource$FlumeHTTPServlet.doPost(HTTPSource.java:201)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:814)
      at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
      at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
      at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
      at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
      at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
      at org.mortbay.jetty.Server.handle(Server.java:326)
      at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
      at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
      at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
      at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
      at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
      at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
      at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
      Caused by: org.apache.flume.ChannelException: Space for commit to queue couldn't be acquired Sinks are likely not keeping up with sources, or the buffer size is too tight
      at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:128)
      at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
      at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:192)
      ... 16 mor16/06/01 08:11:58 WARN http.HTTPSource: Error appending event to channel. Channel might be full. Consider increasing the channel capacity or make sure the sinks perform faster.
      org.apache.flume.ChannelException: Unable to put batch on required channel: org.apache.flume.channel.MemoryChannel

      {name: ckan-channel}

      at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:200)
      at org.apache.flume.source.http.HTTPSource$FlumeHTTPServlet.doPost(HTTPSource.java:201)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:814)
      at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
      at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
      at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
      at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
      at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
      at org.mortbay.jetty.Server.handle(Server.java:326)
      at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
      at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
      at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
      at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
      at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
      at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
      at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
      Caused by: org.apache.flume.ChannelException: Space for commit to queue couldn't be acquired Sinks are likely not keeping up with sources, or the buffer size is too tight
      at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:128)
      at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
      at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:192)
      ... 16 more

      Also I see that hfds-channel/sink is not even active in the logs.

      Is this because ckan-channel is giving errors so that hdfs-channel is not called for this?

        Activity

        Hide
        backlogmanager Backlog Manager added a comment -

        2016-06-02 12:05|CREATED monitor | # answers= 1, accepted answer= True

        Show
        backlogmanager Backlog Manager added a comment - 2016-06-02 12:05|CREATED monitor | # answers= 1, accepted answer= True
        Hide
        backlogmanager Backlog Manager added a comment -

        2016-06-02 15:05|UPDATED status: transition Answer| # answers= 1, accepted answer= True

        Show
        backlogmanager Backlog Manager added a comment - 2016-06-02 15:05|UPDATED status: transition Answer| # answers= 1, accepted answer= True
        Hide
        backlogmanager Backlog Manager added a comment -

        2016-06-02 18:05|UPDATED status: transition Finish| # answers= 1, accepted answer= True

        Show
        backlogmanager Backlog Manager added a comment - 2016-06-02 18:05|UPDATED status: transition Finish| # answers= 1, accepted answer= True

          People

          • Assignee:
            frb Francisco Romero
            Reporter:
            backlogmanager Backlog Manager
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: