Syslog-ng & GNS3

Continuing from the previous NTP post, our GNS3 topology hasn't changed except for UbtSvr:

I created a working syslog-ng server on UbtSvr2 long before I did the NTP post so, to make sure I don't miss a step or forget to mention something I did to get it working, I've gone ahead and created a brand new Ubuntu server in VirtualBox named UbtSvr3. The default route 192.0.0.0/8 and ip address on UbtSvr3 is the same as UbtSvr2 in the previous NTP post. Nothing else has changed.

For this VM (UbtSvr3), I'm using Ubuntu Server 16.04 LTS ran as root (to avoid having to type 'sudo' all the time). Also, to make things easier (for terminal scroll back, window size, a cleaner look, so I can use terminator, etc.) I installed Ubuntu desktop onto the server [ sudo apt-get install ubuntu-desktop ].

Goal
We only want to get syslog files from devices in the GNS3 topology onto UbtSvr3 and that's it. This is just a very basic syslog-ng post using GNS3 and syslog-ng… nothing more.

Install syslog-ng
This new VM doesn't have syslog-ng currently installed:

So I installed it by running:

sudo apt-get install syslog-ng-core

This will install all required files. For me, the installed version is syslog-ng-core_3.5.6-2.1_amd64…

syslog-ng now shows up in /etc:

The configuration file for syslog-ng is: /etc/syslog-ng/syslog-ng.conf

I’m going to back this up in case of any issues:

cp /etc/syslog-ng/syslog-ng.conf  /etc/syslog-ng/syslog-ng.conf.bak

Syslog-ng, goal restated
There are a TON of things you can do with syslog-ng. As mentioned earlier, this post is only interested in getting syslog files from the devices in our topology. But, we don't want just one log file with all logs from all devices in it; we want separate log files, one per device. And that's it, that's our mission, that's all we want to accomplish… at this time. If you want to separate log files by type, add filters or have logs rotated by time / date, I suggest reading the 400+ page manual on syslog-ng; good stuff.

So, after spending some time messing around, this is the syslog-ng.conf configuration I ended up creating:

And Con't…

*At the end of this post is a text version of the syslog-ng.conf you can copy and paste.*

For this post: Ugly is okay - cleanup not important
I've removed all 'Log paths', 'Filters' and most of the 'Destinations' info from the original file and added some other things like a template and variables. I'm sure I could have removed a lot more as this is such a simple configuration (i.e. the only important destination line is d_net, etc.) But the above is what it is and it works; so it shall be for this post. Feel free to mess around all you want - just getting this to work was the hard part, tweaking / breaking it is the fun part!

I've configured this so that messages will be sent to the /var/log/cisco/ folder on UbtSvr3.

I didn't create this folder since syslog-ng should create this automatically ("should"). After restarting syslog-ng and taking a look, it's not there so I went ahead and created it:

**Note, if you restart the syslog-ng service and it fails, go back and carefully look through the syntax - there could be a missing semicolon or something else that's causing it to fail. Usually when it fails it will give a reason which often points to where the issue is.

Now, onto the devices. Checking R1 we can see the only logging is "synchronous", no other configs have been applied:

R1#show run | sec logging
logging synchronous
logging synchronous
R1#

I then added the following to R1:

R1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1(config)#logging trap debugging
R1(config)#logging origin-id hostname
R1(config)#logging 192.168.1.2
R1(config)#service timestamps log uptime
R1(config)#service timestamps log datetime localtime
R1(config)#end
R1#
Jun 17 00:30:27: %SYS-5-CONFIG_I: Configured from console by console
R1#
Jun 17 00:30:28: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 192.168.1.2 port 514 started - CLI initiated
R1#

You can see by the SYS-6 message that logging has started on UDP port 514. I'm going to shut / no shut fa0/0 on R1 (to create syslog messages) and then check UbtSvr3's 'cisco' directory to see if we have any new files:

R1(config-if)#inter fa0/0
R1(config-if)#shut
R1(config-if)#
Jun 17 00:34:00: %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.2 on FastEthernet0/0 from FULL to DOWN, Neighbor Down: Interface down or detached
R1(config-if)#no shut
R1(config-if)#
Jun 17 00:34:02: %LINK-5-CHANGED: Interface FastEthernet0/0, changed state to administratively down
Jun 17 00:34:03: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to down
R1(config-if)#
Jun 17 00:34:06: %LINK-3-UPDOWN: Interface FastEthernet0/0, changed state to up
Jun 17 00:34:07: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to up
R1(config-if)#
Jun 17 00:34:09: %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.2 on FastEthernet0/0 from LOADING to FULL, Loading Done
R1(config-if)#

And…

Cool… it's working - you can see the new '192.168.1.1' file from R1 in /var/log/cisco. Running tail -f 192.168.1.1 on this and doing a shut / no shut on R1's loopback shows continuous logging:

On R1:

R1(config)#inter l0
R1(config-if)#shut
R1(config-if)#
Jun 17 01:51:48: %LINK-5-CHANGED: Interface Loopback0, changed state to administratively down
Jun 17 01:51:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to down
R1(config-if)#no shut
R1(config-if)#
Jun 17 01:51:54: %LINK-3-UPDOWN: Interface Loopback0, changed state to up
Jun 17 01:51:55: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to up
R1(config-if)#shut
R1(config-if)#
Jun 17 01:52:17: %LINK-5-CHANGED: Interface Loopback0, changed state to administratively down
Jun 17 01:52:18: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to down
R1(config-if)#no shut
R1(config-if)#
Jun 17 01:52:23: %LINK-3-UPDOWN: Interface Loopback0, changed state to up
Jun 17 01:52:24: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to up
R1(config-if)#

Output on UbtSvr3:

Adding the same configs to R2, R3 and R4 and, after doing a shut / no shut on their loopback L0 interfaces, there are now log files for each device:

But… the files are named based on the interface they were sent out from. Blah! I think this would look a lot nicer (eh, easier to distinguish for t-shooting) if they showed as the devices hostname instead of their exit IP address.

This can be done a few different ways, namely using a DNS server. But that's a little beyond the scope of this so there's another way - and that's to resolve hostnames locally using the /etc/hosts file.

Here I added the IP to hostname mapping:

Cool, now there's one more thing - in the initial configurations for syslog-ng.conf you may have noticed some lines commented out in the global options:

Namely, the following:

# use_dns(persist_only);
# dns-cache-hosts(/etc/hosts);

I added these but commented them out on purpose. Uncomment these and restart syslog-ng.

Now, after doing a shut / no shut on each devices L0 interface (to get some logging), /var/log/cisco is now showing log files but with each devices hostname:

Blam!
"Mission Accomplished... Yo!"
Farm (GNS3) to table (syslog-ng on UbtSvr3).

I created this post because this is something I've really wanted to do for a while now but couldn't figure out how (until a few weeks ago.) At one point I abandoned syslog-ng and created a Graylog syslog server - that was interesting but not what I really wanted. I wanted log files on a server I could peruse using sed and awk.

There's a tiny bit of information online on how to work with syslog-ng (bits and pieces - mostly inaccurate or outdated) and a couple of YT videos (generic, way outdated and all logs get dumped into one file... not even using syslog-ng). I ended up reading a lot of the syslog-ng manual and that's ultimately how I got this to work (along with some critical thinking and trial / error). Want to figure out how something works? Read the manual!... even if it is in another garbled, translated language. Actually, the manual isn't that bad... mostly just abstract.

Anyway, since this is a working example that can be copied and reproduced, this gives someone the chance to look (understand) how this works, mess with settings (parameters / variables) and have some fun (instead of spending hours and hours w/out success - blah).

If you want to know how the parts of the config I created work - that's the learning part and you should... "read the manual".

Not working?
-Check your ufw rules (firewall rules - make sure udp port 514 is allowed on eth1... or just allow anything on eth1 - make it easy)

-------------
- Sláinte
-------------

As mentioned, below is the text version you can copy and paste (hint: make sure the version, first line, is the same version you are using):

@version: 3.5
@include "scl.conf"
@include "`scl-root`/system/tty10.conf"

# Syslog-ng configuration file, compatible with default Debian syslogd
# installation.

# First, set some global options.

options {

chain_hostnames(on);
flush_lines(0);
use_dns(persist_only);
dns-cache-hosts(/etc/hosts);
use_fqdn(no);
owner("root");
group("adm");
perm(0640);
stats_freq(0);
bad_hostname("^gconfd$");
keep_hostname(yes);

};

########################
# Sources
########################

# This is the default behavior of sysklogd package
# Logs may come from unix stream, but not from another machine.
#

source s_src {

system();
internal();

};

# If you wish to get logs from remote machine you should uncomment
# this and comment the above source line.
#

source s_net { tcp(ip(192.168.1.2) port(1000) max-connections(500)); udp (); };

########################
# Destinations
########################

# First some standard logfile
#

template d_template {
template("${ISODATE} ${FULLHOST_FROM} ${MSG}\n"); };

destination d_net { file("/var/log/cisco/$FULLHOST_FROM" template(d_template)); };
destination d_syslog { file("/var/log/cisco/$FULLHOST_FROM" template(d_template)); };
destination d_net_syslog { file("/var/log/cisco/$FULLHOST_FROM" template(d_template)); };

# Some `catch-all' logfiles.
#

destination d_debug { file("/var/log/cisco" template(d_template)); };
destination d_error { file("/var/log/cisco" template(d_template)); };
destination d_messages { file("/var/log/cisco" template(d_template)); };

# The root's console.
#

destination d_console { usertty("root"); };

# Virtual console.
#

destination d_console_all { file(`tty10`); };

# All messages send to a remote site
#

log { source(s_net); destination(d_net); };

###
# Include all config files in /etc/syslog-ng/conf.d/
###

@include "/etc/syslog-ng/conf.d/*.conf"