Created question in FIWARE Q/A platform on 09-12-2015 at 14:12
Please, ANSWER this question AT http://stackoverflow.com/questions/34180456/why-wont-cygnus-keep-running

Question:
Why wont Cygnus keep running?

Description:
Following the instructions on this link I only changed 2 config files.

The first one is cygnus_instance_1.conf (actually I just renamed the example file):

#####
#

1. Configuration file for apache-flume
   #
   #####
2. Copyright 2014 Telefonica Investigación y Desarrollo, S.A.U
4. This file is part of fiware-cygnus (FI-WARE project).
6. fiware-cygnus is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General
7. Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any
8. later version.
9. fiware-cygnus is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
10. warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
11. details.
13. You should have received a copy of the GNU Affero General Public License along with fiware-cygnus. If not, see
14. http://www.gnu.org/licenses/.
16. For those usages not covered by the GNU Affero General Public License please contact with iot_support at tid dot es

1. Who to run cygnus as. Note that you may need to use root if you want
2. to run cygnus in a privileged port (<1024)
   CYGNUS_USER=cygnus

1. Where is the config folder
   CONFIG_FOLDER=/usr/cygnus/conf

1. Which is the config file
   CONFIG_FILE=/usr/cygnus/conf/agent.conf

1. Name of the agent. The name of the agent is not trivial, since it is the base for the Flume parameters
2. naming conventions, e.g. it appears in .sources.http-source.channels=...
   AGENT_NAME=cygnusagent

1. Name of the logfile located at /var/log/cygnus. It is important to put the extension '.log' in order to the log rotation works properly
   LOGFILE_NAME=cygnus.log

1. Administration port. Must be unique per instance
   ADMIN_PORT=8081

1. Polling interval (seconds) for the configuration reloading
   POLLING_INTERVAL=30

And the second config is agent_1.conf (where I removed all but the mysql DB because that is the one I am using):

1. Copyright 2014 Telefónica Investigación y Desarrollo, S.A.U
3. This file is part of fiware-cygnus (FI-WARE project).
5. fiware-cygnus is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General
6. Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any
7. later version.
8. fiware-cygnus is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
9. warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
10. details.
12. You should have received a copy of the GNU Affero General Public License along with fiware-cygnus. If not, see
13. http://www.gnu.org/licenses/.
15. For those usages not covered by the GNU Affero General Public License please contact with iot_support at tid dot es

#=============================================

1. To be put in APACHE_FLUME_HOME/conf/agent.conf
   #
2. General configuration template explaining how to setup a sink of each of the available types (HDFS, CKAN, MySQL).

#=============================================

1. The next tree fields set the sources, sinks and channels used by Cygnus. You could use different names than the
2. ones suggested below, but in that case make sure you keep coherence in properties names along the configuration file.
3. Regarding sinks, you can use multiple types at the same time; the only requirement is to provide a channel for each
4. one of them (this example shows how to configure 3 sink types at the same time). Even, you can define more than one
5. sink of the same type and sharing the channel in order to improve the performance (this is like having
6. multi-threading).
   cygnusagent.sources = http-source
   cygnusagent.sinks = mysql-sink
   cygnusagent.channels = mysql-channel

#=============================================

1. source configuration
2. channel name where to write the notification events
   cygnusagent.sources.http-source.channels = mysql-channel
3. source class, must not be changed
   cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
4. listening port the Flume source will use for receiving incoming notifications
   cygnusagent.sources.http-source.port = 5050
5. Flume handler that will parse the notifications, must not be changed
   cygnusagent.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.OrionRestHandler
6. URL target
   cygnusagent.sources.http-source.handler.notification_target = /notify
7. Default service (service semantic depends on the persistence sink)
   cygnusagent.sources.http-source.handler.default_service = Trace_Data
8. Default service path (service path semantic depends on the persistence sink)
   cygnusagent.sources.http-source.handler.default_service_path = Sensor
9. Number of channel re-injection retries before a Flume event is definitely discarded (-1 means infinite retries)
   cygnusagent.sources.http-source.handler.events_ttl = 10
10. Source interceptors, do not change
    cygnusagent.sources.http-source.interceptors = ts gi
11. TimestampInterceptor, do not change
    cygnusagent.sources.http-source.interceptors.ts.type = timestamp
12. GroupinInterceptor, do not change
    cygnusagent.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.GroupingInterceptor$Builder
13. Grouping rules for the GroupingInterceptor, put the right absolute path to the file if necessary
14. See the doc/design/interceptors document for more details
    cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file = /usr/cygnus/conf/grouping_rules.conf

1. ============================================
2. OrionMySQLSink configuration
3. channel name from where to read notification events
   cygnusagent.sinks.mysql-sink.channel = mysql-channel
4. sink class, must not be changed
   cygnusagent.sinks.mysql-sink.type = com.telefonica.iot.cygnus.sinks.OrionMySQLSink
   # true if the grouping feature is enabled for this sink, false otherwise
   cygnusagent.sinks.mysql-sink.enable_grouping = false
5. the FQDN/IP address where the MySQL server runs
   cygnusagent.sinks.mysql-sink.mysql_host = 127.0.0.1
6. the port where the MySQL server listes for incomming connections
   cygnusagent.sinks.mysql-sink.mysql_port = 3306
7. a valid user in the MySQL server
   cygnusagent.sinks.mysql-sink.mysql_username = root
8. password for the user above
   cygnusagent.sinks.mysql-sink.mysql_password = klasika
9. how the attributes are stored, either per row either per column (row, column)
   cygnusagent.sinks.mysql-sink.attr_persistence = column
10. select the table type from table-by-destination and table-by-service-path
    cygnusagent.sinks.mysql-sink.table_type = table-by-destination
    # number of notifications to be included within a processing batch
    cygnusagent.sinks.mysql-sink.batch_size = 100
11. timeout for batch accumulation
    cygunsagent.sinks.mysql-sink.batch_timeout = 30

#=============================================

1. mysql-channel configuration
2. channel type (must not be changed)
   cygnusagent.channels.mysql-channel.type = memory
3. capacity of the channel
   cygnusagent.channels.mysql-channel.capacity = 1000
4. amount of bytes that can be sent per transaction
   cygnusagent.channels.mysql-channel.transactionCapacity = 100

#============================================

I even did the test successfully and got the following output:

Tests run: 72, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:28.380s
[INFO] Finished at: Tue Dec 08 15:25:16 CET 2015
[INFO] Final Memory: 43M/112M
[INFO] ------------------------------------------------------------------------

The test worked because I properly configured java and maven:

[root@localhost fiware-cygnus]# echo $JAVA_HOME
/usr/java/jdk1.8.0_65
[root@localhost fiware-cygnus]# java -version
java version "1.8.0_65"
Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)
[root@localhost fiware-cygnus]# mvn -version
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 14:51:28+0100)
Maven home: /usr/local/maven
Java version: 1.8.0_65, vendor: Oracle Corporation
Java home: /usr/java/jdk1.8.0_65/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-504.12.2.el6.x86_64", arch: "amd64", family: "unix"

So all good when I hit service cygnus start because I get:

Starting Cygnus 1... [ OK ]

But then 2 seconds later when I hit service cygnus status I get:

cygnus-flume-ng dead but pid file exists

Context broker is working perfectly by the way, but it just doesnt know that the subscriptions Im entering are not going anywere...

This is the log from cygnus (there is no timestamp so I ill post the first and the last one):

Warning: JAVA_HOME is not set!
+ exec /usr/bin/java -Xmx20m -Dflume.log.file=cygnus.log -cp '/usr/cygnus/conf:/usr/cygnus/lib/:/usr/cygnus/plugins.d/cygnus/lib/:/usr/cygnus/plugins.d/cygnus/libext/*' -Djava.library.path= com.telefonica.iot.cygnus.nodes.CygnusApplic$
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/cygnus/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/cygnus/plugins.d/cygnus/lib/cygnus-0.10.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: ./logs/cygnus.log (No such file or directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:142)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:207)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:809)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735)
at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
at org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:73)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:242)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:254)
at org.apache.flume.node.Application.<clinit>(Application.java:58)

Warning: JAVA_HOME is not set!
+ exec /usr/bin/java -Xmx20m -Dflume.log.file=cygnus.log -cp '/usr/cygnus/conf:/usr/cygnus/lib/:/usr/cygnus/plugins.d/cygnus/lib/:/usr/cygnus/plugins.d/cygnus/libext/*' -Djava.library.path= com.telefonica.iot.cygnus.nodes.CygnusApplic$
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/cygnus/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/cygnus/plugins.d/cygnus/lib/cygnus-0.10.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: ./logs/cygnus.log (No such file or directory)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at java.io.FileOutputStream.<init>(FileOutputStream.java:133)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:207)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:809)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735)
at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
at org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:73)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:242)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:254)
at org.apache.flume.node.Application.<clinit>(Application.java:58)

I demonstrated that I have the latest version of java running and that its home env is set. What am I doing wrong here? I should stress that I havent touched any other config files what so ever. Here is the full list that I found:

[root@localhost conf]# ls
agent_1.conf cygnus_instance_1.conf flume-env.sh.template grouping_rules.conf.template krb5_login.conf log4j.properties.template
agent.conf.template cygnus_instance.conf.template grouping_rules.conf krb5.conf.template log4j.properties README.md

If you are not familiar with FIWARE technologies, please ignore this question!