19
Fri, Apr
5 New Articles

TechTip: Take Advantage of Linux Remote System Logging

Typography
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

Centralized remote logging can be directly utilized with Linux's built-in syslog logging facilities.

 

An important part of Linux system administration is dealing with system logs. Even if you aren't watching or reading logs on a daily basis, when you're troubleshooting problems, logs are usually the very first thing you turn to when troubles arise. Having logs in one centralized location on a network simplifies the task of locating them, plus gives you one place to create backups of logs if desired. The default Linux logging facility, syslog, provides you with the basic tools necessary to send logs to a remote server for collection.

The syslog Daemon

The syslog daemon is a standard tool that comes with all Linux flavors. It uses UDP network protocol and runs on port 514. Virtually all log files are found under /var/log on Linux systems, although in rare circumstances you can find logs elsewhere. Red Hat, CentOS, Debian, and SuSE are just a few examples of where configuration and use are virtually identical on all Linux variants. Before you start editing the syslog configuration files, let's spend a few minutes understanding basic syslog terms, something often overlooked. Open up /etc/syslog.conf with your favorite text editor to see the default logging options.

 

The first field is called a facility and basically is the name of the system process where the log will be coming from. The following lists some facility names with their system process definitions. More do exist, but these are the most commonly seen facilities.

 

  • kern—All kernel-related messages
  • authpriv (formerly auth)—Security-related messages
  • mail—Mail system messages
  • cron—Clock and scheduled job messages
  • daemon—System daemon messages
  • user—User processes
  • local0 to local7—User-defined facilities

 

The next piece configured directly after the facility name is the priority-level options. This list explains the various levels of priority logging.

 

  • none—Used to disable a facility
  • info—Information message
  • warning (formerly warn)—Warning messages
  • debug—Debug type of message, usually verbose
  • crit—Critical messages
  • err (formerly error)—Error-related messages
  • alert—Immediate action is requested
  • emerg (formerly panic)—Extremely problematic messages

 

The last configuration parameter is the actual location to send the log files to. To send to a file, an option would be listed as something like /var/log/message or /var/log/any_file_name. The asterisk (*) option is used as a placeholder with a meaning of "all." The default authpriv definition listed in /etc/syslog.conf is shown below.

 

authpriv.*              /var/log/secure

 

This definition states that the authpriv facility and all priority log-level messages should be sent to the file /var/log/secure. Another more complicated line is shown here.

 

*.info;mail.none;authpriv.none;cron.none        /var/log/messages

 

By using the * placeholder and some .none definitions, you can include and exclude facilities and priorities. This statement means that syslog selects all informational messages except those from mail, authpriv, and cron facilities and sends the logs to /var/log/messages.

 

These are just the basics and are meant to get you started, so if want a lot more detailed information, do some Web searches for syslog. Also, you can visit the syslog Web site, which has some excellent resources.

Enable Remote Logging

Syslog's default configuration has the remote logging features turned off completely. Open up the syslog daemon configuration file at /etc/sysconfig/syslog and add the remote logging feature. The option -r is used to specify remote logging. This option need only be changed on the central syslog server. All of the syslog clients will point to this one server.

 

SYSLOGD_OPTIONS="-r -m 0"

 

Once the option is changed, restart the syslog daemon for the changes to take effect. On Red Hat and CentOS systems, the service command is used. On many other systems, the path to the syslog executable can be used.

 

# service syslog restart

# /etc/init.d/syslog restart

 

Pointing Clients to the Remote Server

On the client machines from where you want the logs sent to the remote syslog server, open the syslog configuration file at /etc/syslog.conf. Everywhere that there is a path to a file or on every logging facility that you want sent to the remote server, uncomment out the default and add either the IP address or the DNS name of the remote syslog server.

 

#authpriv.*            /var/log/secure

 

authpriv.*              @192.168.0.1

 

After the changes are made, restart the syslog server the same as you did above on the syslog server. Once restarted, the logs that you pointed to the remote syslog server should start to show up on the remote server's file. In the above example, the authpriv facility should have its logs passed on to the IP address of 192.168.0.1. Since SSH runs under this facility, attempt to log in to the client from a new machine, while viewing the logs on the remote syslog server.

 

On the remote syslog server, run the tail command to view the log in action.

 

# tail -f /var/log/secure

 

Now, from a new machine, attempt to log in using SSH to the remote client you just set up to send logs to the remote syslog server.

 

# ssh user@remote_client

 

With the log file open on the remote syslog server, you should see some activity based upon either an IP address or a DNS name from where you're logging into. Once you exit the SSH session, you should see a disconnect message as well.

 

Sep 14 11:26:13 remote_client sshd[9072]: pam_unix(sshd:session): session opened for user root by (uid=0)

 

Sep 14 11:26:49 remote_client sshd[9072]: pam_unix(sshd:session): session closed for user root

 

Firewall Rules and Security

You may run into some firewall rules problems from time to time. If you're going to use remote syslog features on a trusted local network, chances are there won't be any firewalls in the road. If, however, there's a firewall on the remote syslog server, you have options to add (by IP addresses or by subnets) to trust the traffic being passed to the server. You should always consult your network administrator to see what the policies are at your location.

 

If you have a concern about security, you may want to look into rsyslog. The rsyslog tool is the new and improved version of syslog. Its main focus is on reliability and security, and it allows transport of logs over TCP using SSL and TLS security protocols.

Remote Troubleshooting

Something as simple as a centralized place to store logs may seem trivial, but it's very important. Having one place to start troubleshooting can save you time. Plus, if a machine or server has become completely unresponsive, the logs leading up to the crash won't be on the troubled hardware, and instead will be on a remote location for you to view.

 

 

Max Hetrick

Max Hetrick is an Information Systems Assistant for an electric utility. He has experience with installation and maintenance of both Windows and Linux operating systems from the PC to server levels. Max is also an open-source software advocate. He welcomes all comments and can be reached at This email address is being protected from spambots. You need JavaScript enabled to view it..

BLOG COMMENTS POWERED BY DISQUS

LATEST COMMENTS

Support MC Press Online

$0.00 Raised:
$

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: