Hi Miguel,

Sorry for the late reply, I was on holidays.

That kind of error appears when the data producer (the NGSI-like notifications originator) is faster than the data consumer (Cygnus's sinks). Please consider this guidelines for improving the performance:
https://github.com/telefonicaid/fiware-cygnus/blob/master/doc/operation/performance_tuning_tips.md

In addition, if you are not using the Grouping feature, simply avoid that part in the configuration (even when there are no grouping rules, the notified events are iterated if configured the interceptor):

cygnusagent.sources.http-source.interceptors = ts

cygnusagent.sources.http-source.interceptors.ts.type = timestamp

1. the following lines are not necessary

#cygnusagent.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.GroupingInterceptor$Builder

#cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file = /usr/cygnus/conf/grouping_rules.conf

Finally, we are considering the usage of "binary" APIs instead of REST-based ones when dealing with the backends in order to improve the performance, please check:
https://github.com/telefonicaid/fiware-cygnus/issues/301

Hope this helps. Regards,
Francisco

De: MAPT Talkykar <miguel@talkykar.com<miguel@talkykar.com>>
Fecha: miércoles, 2 de septiembre de 2015, 9:39
Para: Francisco Romero Bueno <francisco.romerobueno@telefonica.com<francisco.romerobueno@telefonica.com>>, Pasquale Vitale <pvitale@eng.it<pvitale@eng.it>>
Asunto: Problems with Cygnus v0.8.2 - Error appending event to channel. Channel might be full

Hello

We are having troubles with Cygnus. When we have some devices sending information to Context Broker (and context broker subscripted to Cygnus, sending information to Cosmos and CKAN), the log show errors

There are errors both ckan-channel and hdfs-channel

Here a subset of log.

02 Sep 2015 06:46:07,948 WARN [1701535537@qtp-1850408381-327] (org.apache.flume.source.http.HTTPSource$FlumeHTTPServlet.doPost:203) - Error appending event to channel. Channel might be full. Consider increasing the channel capacity or make sure the sinks perform faster.
org.apache.flume.ChannelException: Unable to put batch on required channel: org.apache.flume.channel.MemoryChannel

at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:200)
at org.apache.flume.source.http.HTTPSource$FlumeHTTPServlet.doPost(HTTPSource.java:201)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:814)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: org.apache.flume.ChannelException: Space for commit to queue couldn't be acquired Sinks are likely not keeping up with sources, or the buffer size is too tight
at org.apache.flume.channel.MemoryChannel

The complete log is available here:
https://dl.dropboxusercontent.com/u/2537618/cygnus_a.log

–
https://docs.google.com/uc?export=download&id=0B7AOtUDEGtBMbEdoMkZybzNnV2s&revid=0B7AOtUDEGtBML3lzZitYd1RqZU5VNGFKRDJzRzZDZFh3UEZjPQ

________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição

_______________________________________________
Fiware-finodex-coaching mailing list
Fiware-finodex-coaching@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-finodex-coaching-new
[Created via e-mail received from: FRANCISCO ROMERO BUENO <francisco.romerobueno@telefonica.com>]