鸿 网 互 联 www.68idc.cn

Linux下rsyslog日志收集服务环境部署记录

来源:互联网 作者:佚名 时间:2022-07-19 11:12
rsyslog 可以理解为多线程增强版的syslog。 在syslog的基础上扩展了很多其他功能,如数据库支持(MySQL、PostgreSQL、Oracle等)、日志内容筛选、定义日志格式模板等。目前大多数Linux发行版

rsyslog 可以理解为多线程增强版的syslog。 在syslog的基础上扩展了很多其他功能,如数据库支持(MySQL、PostgreSQL、Oracle等)、日志内容筛选、定义日志格式模板等。目前大多数Linux发行版默认也是使用rsyslog进行日志记录。rsyslog提供了三种远程传输协议:

UDP?传输协议? 基于传统UDP协议进行远程日志传输,也是传统syslog使用的传输协议;?可靠性比较低,但性能损耗最少,?在网络情况比较差,?或者接收服务器压力比较高情况下, 可能存在丢日志情况。?在对日志完整性要求不是很高,在可靠的局域网环境下可以使用。 TCP?传输协议? 基于传统TCP协议明文传输,需要回传进行确认,可靠性比较高;?但在接收服务器宕机或者两者之间网络出问题的情况下,会出现丢日志情况。?这种协议相比于UDP在 可靠性方面已经好很多,并且rsyslog原生支持,配置简单,?同时针对可能丢日志情况,可以进行额外配置提高可靠性,因此使用比较广。 RELP?传输协议? RELP(Reliable?Event?Logging?Protocol)是基于TCP封装的可靠日志消息传输协议;?是为了解决TCP?与?UDP?协议的缺点而在应用层实现的传输协议,也是三者 之中最可靠的。?需要多安装一个包rsyslog-relp以支持该协议。 对于线上服务器,为了日志安全起见,建议使用还是使用?RELP?协议进行传输。

rsyslog的简单配置记录(如下将公司防火墙上的日志(UDP)打到IDC的rsyslog日志服务器上)

一、rsyslog服务端的部署 安装rsyslog?程序(rsyslog默认已经在各发行版安装,如果系统中没有的话,可以用yum?进行安装,如下:) [root@zabbix?~]#?yum?install?rsyslog?-y 配置: [root@zabbix?~]#?cat?/etc/rsyslog.conf #?rsyslog?v5?configuration?file #?For?more?information?see?/usr/share/doc/rsyslog-*/rsyslog_conf.html #?If?you?experience?problems,?see?http://www.rsyslog.com/doc/troubleshoot.html ####?MODULES?#### $ModLoad?imuxsock?#?provides?support?for?local?system?logging?(e.g.?via?logger?command) $ModLoad?imklog???#?provides?kernel?logging?support?(previously?done?by?rklogd) $ModLoad?immark??#?provides?--MARK--?message?capability #?Provides?UDP?syslog?reception $ModLoad?imudp??????????????????????????????????????????#开启udp的514端口。也可以开启tcp的514端口,这里只接受udp的 $UDPServerRun?514 #?Provides?TCP?syslog?reception #$ModLoad?imtcp #$InputTCPServerRun?514 $WorkDirectory?/var/lib/rsyslog $AllowedSender?udp,?192.168.17.0/8????????????????????#仅仅接收来自192.168.17.0/8网段的主机的udp日志(这个是公司防火墙的ip地址) ####?GLOBAL?DIRECTIVES?#### #?Use?default?timestamp?format $ActionFileDefaultTemplate?RSYSLOG_TraditionalFileFormat $template?Remote,"/data/fw_logs/%fromhost-ip%/%fromhost-ip%_%$YEAR%-%$MONTH%-%$DAY%.log"???????????#定义模板,接受日志文件路径,区分了不同主机的日志 :fromhost-ip,?!isequal,?"127.0.0.1"??Remote????????????????????????????????????????????????????????#?过滤server?本机的日志 #?File?syncing?capability?is?disabled?by?default.?This?feature?is?usually?not?required, #?not?useful?and?an?extreme?performance?hit #$ActionFileEnableSync?on #?Include?all?config?files?in?/etc/rsyslog.d/ $IncludeConfig?/etc/rsyslog.d/*.conf ####?RULES?#### #?Log?all?kernel?messages?to?the?console. #?Logging?much?else?clutters?up?the?screen. #kern.*?????????????????????????????????????????????????/dev/console #?Log?anything?(except?mail)?of?level?info?or?higher. #?Don't?log?private?authentication?messages! *.info;mail.none;authpriv.none;cron.none????????????????/var/log/messages #?The?authpriv?file?has?restricted?access. authpriv.*??????????????????????????????????????????????/var/log/secure #?Log?all?the?mail?messages?in?one?place. mail.*??????????????????????????????????????????????????-/var/log/maillog local4.*????????????????????????????????????????????????/data/fw.log #?Log?cron?stuff cron.*??????????????????????????????????????????????????/var/log/cron #?Everybody?gets?emergency?messages *.emerg?????????????????????????????????????????????????* #?Save?news?errors?of?level?crit?and?higher?in?a?special?file. uucp,news.crit??????????????????????????????????????????/var/log/spooler #?Save?boot?messages?also?to?boot.log local7.*????????????????????????????????????????????????/var/log/boot.log #?###?begin?forwarding?rule?### #?The?statement?between?the?begin?...?end?define?a?SINGLE?forwarding #?rule.?They?belong?together,?do?NOT?split?them.?If?you?create?multiple #?forwarding?rules,?duplicate?the?whole?block! #?Remote?Logging?(we?use?TCP?for?reliable?delivery) # #?An?on-disk?queue?is?created?for?this?action.?If?the?remote?host?is #?down,?messages?are?spooled?to?disk?and?sent?when?it?is?up?again. #$WorkDirectory?/var/lib/rsyslog?#?where?to?place?spool?files #$ActionQueueFileName?fwdRule1?#?unique?name?prefix?for?spool?files #$ActionQueueMaxDiskSpace?1g???#?1gb?space?limit?(use?as?much?as?possible) #$ActionQueueSaveOnShutdown?on?#?save?messages?to?disk?on?shutdown #$ActionQueueType?LinkedList???#?run?asynchronously #$ActionResumeRetryCount?-1????#?infinite?retries?if?host?is?down #?remote?host?is:?name/ip:port,?e.g.?192.168.0.1:514,?port?optional #*.*?@@remote-host:514 #?###?end?of?the?forwarding?rule?### [root@zabbix?~]#?mkdir?/data/fw_logs/ [root@zabbix?~]#?/etc/init.d/rsyslog?restart 二、在公司防火墙(192.168.17.41/42)上配置udp日志输出策略(在防火墙添加rsyslog服务端的ip和514端口) 三、过一会儿,在rsyslog日志服务器上设置的日志目录下就能看到防火墙的日志输出了 [root@zabbix?~]#?ll?/data/fw_logs/ total?4.0K drwxrwxrwx???4?root?root???46?Jul?28?10:40?. drwxr-xr-x.?18?root?root?4.0K?Jul?28?10:38?.. drwx------???2?root?root???41?Jul?28?10:37?192.168.17.41 drwx------???2?root?root???41?Jul?28?10:40?192.168.17.42 [root@zabbix?~]#?ll?/data/fw_logs/192.168.17.41 total?16K drwx------?2?root?root??41?Jul?28?10:37?. drwxrwxrwx?4?root?root??46?Jul?28?10:40?.. -rw-------?1?root?root?13K?Jul?28?14:02?192.168.17.41_2017-07-28.log ------------------------------------------------------------------------------------ 可以将上面rsyslog服务端的rsyslog.conf里的ip白名单设置为客户机的ip端,比如: $AllowedSender?tcp,?172.18.0.0/16??????????????????#表示接收172.18.0.0/16网段的客户机的tcp日志输入,前提是打开tcp的514端口 客户机的配置: 只需要在rsyslog.conf文件里添加下面一行: *.*???????????????????????????????@172.18.10.20?????????????????????#后面的ip是rsyslog服务端的ip地址 启动rsyslog日志即可!

====================再看一例=======================
以上配置的是将公司防火墙的日志打到rsyslog里。现在有这么一个需求:
公司IDC的另外两台服务器172.19.10.24和172.19.10.25上部署了gitlab、nexus、jenkins、jira和wiki,上面的权限设置的比较杂,很多人都有登录需求。现在需要将登录到这两台服务器上的用户的所有操作过程记录下来,记录达到rsyslog日志里,相当于做用户操作记录的审计工作。

配置如下(结合上面的安装配置)(服务端的ip是172.19.16.21): 1)rsyslog服务端配置??(相比于上面的配置,这里去掉了AllowedSender的来源ip的白名单限制。即允许接收所有机器的日志;上面的防火墙日志还是能继续收集) [root@zabbix?~]#?cat?/etc/rsyslog.conf|grep?-v?"#"|grep?-v?"^$" $ModLoad?imudp $UDPServerRun?514 $WorkDirectory?/var/lib/rsyslog $ActionFileDefaultTemplate?RSYSLOG_TraditionalFileFormat $template?Remote,"/data/fw_logs/%fromhost-ip%/%fromhost-ip%_%$YEAR%-%$MONTH%-%$DAY%.log" :fromhost-ip,?!isequal,?"127.0.0.1"??Remote $IncludeConfig?/etc/rsyslog.d/*.conf *.info;mail.none;authpriv.none;cron.none????????????????/var/log/messages authpriv.*??????????????????????????????????????????????/var/log/secure mail.*??????????????????????????????????????????????????-/var/log/maillog cron.*??????????????????????????????????????????????????/var/log/cron *.emerg?????????????????????????????????????????????????* uucp,news.crit??????????????????????????????????????????/var/log/spooler local7.*????????????????????????????????????????????????/var/log/boot.log local5.*??????????????????????????????????????????????/var/log/history.log [root@zabbix?~]#?/etc/init.d/rsyslog?restart 2)在172.19.10.24上的配置 [root@gitlab?~]#?cat?/etc/rsyslog.conf|grep?-v?"#"|grep?-v?"^$" $ActionFileDefaultTemplate?RSYSLOG_TraditionalFileFormat $IncludeConfig?/etc/rsyslog.d/*.conf *.info;mail.none;authpriv.none;cron.none????????????????/var/log/messages authpriv.*??????????????????????????????????????????????/var/log/secure mail.*??????????????????????????????????????????????????-/var/log/maillog cron.*??????????????????????????????????????????????????/var/log/cron *.emerg?????????????????????????????????????????????????* uucp,news.crit??????????????????????????????????????????/var/log/spooler local7.*????????????????????????????????????????????????/var/log/boot.log local5.*????@172.19.16.21 [root@gitlab?~]#?/etc/init.d/rsyslog?restart [root@gitlab?~]#?cat?/etc/profile??????????????????#在该文件的底部添加下面内容 ....... export?HISTTIMEFORMAT export?PROMPT_COMMAND='{?command=$(history?1?|?{?read?x?y;?echo?$y;?});?logger?-p?local5.notice?-t?bash?-i?"user=$USER,ppid=$PPID,from=$SSH_CLIENT,pwd=$PWD,command:$command";?}' 3)在另一台172.19.10.25上做类似配置配置 [root@nexus?~]#?cat?/etc/rsyslog.conf?|grep?-v?"#"|grep?-v?"^$" $ActionFileDefaultTemplate?RSYSLOG_TraditionalFileFormat $IncludeConfig?/etc/rsyslog.d/*.conf *.info;mail.none;authpriv.none;cron.none????????????????/var/log/messages authpriv.*??????????????????????????????????????????????/var/log/secure mail.*??????????????????????????????????????????????????-/var/log/maillog cron.*??????????????????????????????????????????????????/var/log/cron *.emerg?????????????????????????????????????????????????* uucp,news.crit??????????????????????????????????????????/var/log/spooler local7.*????????????????????????????????????????????????/var/log/boot.log local5.*???@172.19.16.21? [root@nexus?~]#?/etc/init.d/rsyslog?restart [root@nexus?~]#?cat?/etc/profile ....... export?HISTTIMEFORMAT export?PROMPT_COMMAND='{?command=$(history?1?|?{?read?x?y;?echo?$y;?});?logger?-p?local5.notice?-t?bash?-i?"user=$USER,ppid=$PPID,from=$SSH_CLIENT,pwd=$PWD,command:$command";?}' 4)过一段时间,发现在rsyslog服务端的日志目录/data/fw_logs下面已经有收集到的日志了 [root@zabbix?fw_logs]#?pwd /data/fw_logs [root@zabbix?fw_logs]#?cd [root@zabbix?~]#?cd?/data/fw_logs/ [root@zabbix?fw_logs]#?ll total?12K drwxrwxrwx???6?root?root???84?Aug?16?18:28?. drwxr-xr-x.?18?root?root?4.0K?Aug?16?17:58?.. drwx------???2?root?root???74?Aug?17?09:50?172.19.10.24 drwx------???2?root?root???74?Aug?17?10:00?172.19.10.25 drwx------???2?root?root?4.0K?Aug?17?00:01?192.168.17.41 drwx------???2?root?root?4.0K?Aug?17?00:01?192.168.17.42 [root@zabbix?fw_logs]#?cd?172.19.10.24/ [root@zabbix?172.19.10.24]#?ll total?20K drwx------?2?root?root??74?Aug?17?09:50?. drwxrwxrwx?6?root?root??84?Aug?16?18:28?.. -rw-------?1?root?root?14K?Aug?16?20:45?172.19.10.24_2017-08-16.log -rw-------?1?root?root?771?Aug?17?10:03?172.19.10.24_2017-08-17.log [root@zabbix?172.19.10.24]#?cat?172.19.10.24_2017-08-16.log Aug?16?18:39:56?gitlab?bash[138413]:?user=root,ppid=124297,from=172.19.16.28?29338?22,pwd=/root,command:[2017-08-16?18:39:56]root?pts/5?2017-08-16?17:23?(172.19.16.28)/etc/init.d/rsyslog?restart Aug?16?18:39:56?gitlab?bash[138418]:?user=root,ppid=124297,from=172.19.16.28?29338?22,pwd=/root,command:[2017-08-16?18:39:56]root?pts/5?2017-08-16?17:23?(172.19.16.28)/etc/init.d/rsyslog?restart Aug?16?18:39:56?gitlab?bash[138422]:?user=root,ppid=124297,from=172.19.16.28?29338?22,pwd=/root,command:[2017-08-16?18:39:56]root?pts/5?2017-08-16?17:23?(172.19.16.28)/etc/init.d/rsyslog?restart Aug?16?18:39:57?gitlab?bash[138426]:?user=root,ppid=124297,from=172.19.16.28?29338?22,pwd=/root,command:[2017-08-16?18:39:56]root?pts/5?2017-08-16?17:23?(172.19.16.28)/etc/init.d/rsyslog?restart Aug?16?18:40:30?gitlab?bash[138610]:?user=root,ppid=138586,from=172.16.255.202?52496?22,pwd=/root,command:[2017-08-16?18:40:03]root?pts/0?2017-08-16?18:40?(172.16.255.202)exit Aug?16?18:40:43?gitlab?bash[138652]:?user=root,ppid=138586,from=172.16.255.202?52496?22,pwd=/data,command:[2017-08-16?18:40:43]root?pts/0?2017-08-16?18:40?(172.16.255.202)cd?/data/ Aug?16?18:40:43?gitlab?bash[138657]:?user=root,ppid=138586,from=172.16.255.202?52496?22,pwd=/data,command:[2017-08-16?18:40:43]root?pts/0?2017-08-16?18:40?(172.16.255.202)ls Aug?16?18:40:47?gitlab?bash[138666]:?user=root,ppid=138586,from=172.16.255.202?52496?22,pwd=/data,command:[2017-08-16?18:40:47]root?pts/0?2017-08-16?18:40?(172.16.255.202)mkdir?hahahahah Aug?16?18:40:48?gitlab?bash[138671]:?user=root,ppid=138586,from=172.16.255.202?52496?22,pwd=/data/hahahahah,command:[2017-08-16?18:40:48]root?pts/0?2017-08-16?18:40?(172.16.255.202)cd?hahahahah/ Aug?16?18:40:48?gitlab?bash[138677]:?user=root,ppid=138586,from=172.16.255.202?52496?22,pwd=/data/hahahahah,command:[2017-08-16?18:40:48]root?pts/0?2017-08-16?18:40?(172.16.255.202)ls Aug?16?18:40:54?gitlab?bash[138696]:?user=root,ppid=138586,from=172.16.255.202?52496?22,pwd=/data/hahahahah,command:[2017-08-16?18:40:54]root?pts/0?2017-08-16?18:40?(172.16.255.202)echo?"Asdfasdf"?>heihei Aug?16?18:40:54?gitlab?bash[138702]:?user=root,ppid=138586,from=172.16.255.202?52496?22,pwd=/data/hahahahah,command:[2017-08-16?18:40:54]root?pts/0?2017-08-16?18:40?(172.16.255.202)ls ....... 有上面日志可以看出,在172.19.10.24这台机器上的操作记录都被详细记录下来了。这样,就能清楚地知道登录到这台机器上的用户都做了些什么了.......

=====================通过rsyslog收集nginx日志到远程服务器上====================
需求说明:通过rsyslog服务将192.168.10.21服务器上的/data/nginx/logs/www.kevin.com-access.log日志实时同步到192.168.10.52服务器上(路径为/data/rsyslog/nginx)。

1)192.168.10.21为rsyslog客户端,即日志的推送端。rsyslog日志是客户机主动将自己的日志推送到远程服务器上。
操作如下:
[root@nginx-server ~]# yum install rsyslog -y
[root@nginx-server ~]# cp /etc/rsyslog.conf /etc/rsyslog.conf.bak
[root@nginx-server ~]# cat /etc/rsyslog.conf
# rsyslog v5 configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog ? # provides kernel logging support (previously done by rklogd)
#$ModLoad immark ?# provides --MARK-- message capability
$ModLoad imfile? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?##装载imfile模块,这一行手动添加

# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514


#### GLOBAL DIRECTIVES ####

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf


#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none;local5.none ? ? ? ? ? ? ? ?/var/log/messages? ? ? ? ? ? ?##不记录local5的日志

# The authpriv file has restricted access.
authpriv.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/secure

# Log all the mail messages in one place.
mail.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?-/var/log/maillog


# Log cron stuff
cron.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/cron

# Everybody gets emergency messages
*.emerg ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? *

# Save news errors of level crit and higher in a special file.
uucp,news.crit ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/spooler

# Save boot messages also to boot.log
local7.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/boot.log


# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$WorkDirectory /var/lib/rsyslog # where to place spool files
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g ? # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList ? # run asynchronously
#$ActionResumeRetryCount -1 ? ?# infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###
user.info /var/log/history

#在文件底部添加下面几行内容
$InputFileName /data/nginx/logs/www.kevin.com-access.log? ? ? ? ##读取日志文件(要监控的日志文件)
$InputFileTag web_access? ? ? ? ? ? ?##日志写入日志附加标签字符串
$InputFileSeverity info? ? ? ? ? ?##日志等级
$InputFileStateFile /etc/rsyslog.d/stat-access? ? ? ? ?##记录日志点等信息。(相当于msyql的master.info)文件名变了,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 这个StateFile标志必须变,否则无法传输。
$InputFileFacility local5? ? ? ? ?##设施类别
$InputFilePollInterval 1? ? ? ? ? ##检查日志文件间隔(秒)
$InputFilePersistStateInterval 1? ? ? ?##回写偏移量数据到文件间隔时间(秒)
$InputRunFileMonitor? ? ? ? ? ? ? ? ? ? ? ? ? ##激活读取,可以设置多组日志读取,每组结束时设置本参数。以示生效。
local5.* ?@192.168.10.52? ? ? ? ? ? ##代表local5设施的所有级别通过udp协议传送到192.168.10.51

重启rsyslog服务
[root@nginx-server ~]# /etc/init.d/rsyslog restart
关闭系统日志记录器: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [确定]
启动系统日志记录器: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [确定]

由于作为日志的推送端,rsyslog日志不需要开启514端口(如上在rsyslog.conf文件里没有打开dup或tcp的514端口)
[root@nginx-server ~]# lsof -i:514
[root@nginx-server ~]#

2)192.168.10.52为rsyslog服务端,即日志的接收端。
配置如下:
[root@log-server ~]# yum install rsyslog -y
[root@log-server ~]# cp /etc/rsyslog.conf /etc/rsyslog.conf.bak
# rsyslog configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
#$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark ?# provides --MARK-- message capability

# Provides UDP syslog reception
$ModLoad imudp? ? ? ? ? ? ? ? ? ?##载入imudp模块
$UDPServerRun 514? ? ? ? ? ? ##开启udp接收并制定端口号

# Provides TCP syslog reception
$ModLoad imtcp? ? ? ? ? ? ? ? ?##载入imtcp模块。
$InputTCPServerRun 514? ? ? ? ? ? ?##开启tcp接收并制定端口号。tcp和udp两个端口模块可以同时使用!

#### GLOBAL DIRECTIVES ####

# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

#定义一个模板用来指定接收的日志消息的格式(默认会在记录的日志前加几个字段)
$template? SpiceTmpl,"%msg%\n"? ? ? ? ? ? ? ? ? ?##%msg:2:$%为去掉日志开头的空格

#定义一个模板用来指定接收的日志文件的存放路径%……%之间的是定义日志按照年-月-日命名
$template? DynaFile,"/data/rsyslog/nginx/%$YEAR%-%$MONTH%-%$DAY%.log"

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on

# File to store the position in the journal
$IMJournalStateFile imjournal.state


#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none;local5.none? ? ? ? ? ? ? ? /var/log/messages? ? ? ? ? ??##不记录local5设施的日志

# The authpriv file has restricted access.
authpriv.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/secure

# Log all the mail messages in one place.
mail.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?-/var/log/maillog


# Log cron stuff
cron.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/cron

# Everybody gets emergency messages
*.emerg ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? :omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/spooler

# Save boot messages also to boot.log
local7.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/boot.log

#接收客户端local5设施传送来的日志并存放到指定位置(位置可用定义的模板。?代表使用动态的模板)
local5.*? ? ? ? ? ? ? ? ? ? ? ??DynaFile;SpiceTmpl

# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g ? # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList ? # run asynchronously
#$ActionResumeRetryCount -1 ? ?# infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###

编辑/etc/sysconfig/rsyslog中"SYSLOGD_OPTIONS="开启远程日志接收功能
[root@log-server ~]# cat /etc/sysconfig/rsyslog
# Options for rsyslogd
# Syslogd options are deprecated since rsyslog v3.
# If you want to use them, switch to compatibility mode 2 by "-c 2"
# See rsyslogd(8) for more details
SYSLOGD_OPTIONS="-c 5"

创建日志接收过来后定义的存放目录
[root@log-server ~]# mkdir -p /data/rsyslog/nginx

重启rsyslog服务
[root@log-server ~]# /etc/init.d/rsyslog restart
Shutting down system logger: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [ ?OK ?]
Starting system logger: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[ ?OK ?]
[root@log-server ~]# lsof -i:514
COMMAND ? ?PID USER ? FD ? TYPE ? DEVICE SIZE/OFF NODE NAME
rsyslogd 24594 root ? ?2u ?IPv4 38927639 ? ? ?0t0 ?TCP *:shell (LISTEN)
rsyslogd 24594 root ? ?3u ?IPv4 38927635 ? ? ?0t0 ?UDP *:syslog
rsyslogd 24594 root ? ?4u ?IPv6 38927636 ? ? ?0t0 ?UDP *:syslog
rsyslogd 24594 root ? ?5u ?IPv6 38927640 ? ? ?0t0 ?TCP *:shell (LISTEN)

查看日志是否接收过来了
[root@log-server ~]# ll /data/rsyslog/nginx/
total 550876
-rw------- 1 root root 483539594 Jun 13 12:58 2018-06-13.log
[root@log-server ~]# tail -2 /data/rsyslog/nginx/2018-06-13.log
1.203.163.198 - [27/Apr/2018:00:17:53 +0800] "POST /scf/%7B%7BloginConfig.loginSubmitUrl%7D%7D HTTP/1.1" 302 0 "https://www.kevin.com/scf/login" Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.62 Safari/537.36 - 0.010 0.003 10.0.54.21:9020 302
1.203.163.198 - [27/Apr/2018:00:17:53 +0800] "POST /scf/%7B%7BloginConfig.loginSubmitUrl%7D%7D HTTP/1.1" 302 0 "https://www.kevin.com/scf/login" Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.62 Safari/537.36 - 0.012 0.003 10.0.54.21:9020 302

=========================温馨提示========================
rsyslog也可以收集多个日志文件,需要注意的是:
$InputFileTag? ? ? ? 定义的APPNAME必须唯一,同一台主机上不同的应用应当使用不同的APPNAME,否则会导致新定义的TOKEN和TAG不生效;
$template? ? ? ? ?定义的模板名必须唯一,否则会导致新定义的TOKEN和TAG不生效;
$InputFileStateFile? ? ? ?定义的StateFile必须唯一,它被rsyslog用于记录文件上传进度,否则会导致混乱;

如下是rsyslog收集多个日志的配置,这里以2个日志文件为例:

日志的推送端配置

[root@external-lb01?~]#?cat?/etc/rsyslog.conf .......... $ModLoad?imfile ......... *.info;mail.none;authpriv.none;cron.none;local5.none;local4.none????????????????/var/log/messages ......... $InputFileName?/data/nginx/logs/portal.kevin.com-access.log $InputFileTag?portal_access $InputFileSeverity?info $InputFileStateFile?/etc/rsyslog.d/stat1-access $InputFileFacility?local4 $InputFilePollInterval?1 $InputFilePersistStateInterval?1 $InputRunFileMonitor? local4.*??@192.168.10.52 $InputFileName?/data/nginx/logs/www.kevin.com-access.log $InputFileTag?web_access $InputFileSeverity?info $InputFileStateFile?/etc/rsyslog.d/stat-access $InputFileFacility?local5 $InputFilePollInterval?1 $InputFilePersistStateInterval?1 $InputRunFileMonitor local5.*??@192.168.10.52 重启日志发送端的rsyslog服务 [root@external-lb01?~]#?/etc/init.d/rsyslog?restart

日志的接收端配置

[root@open-falcon01?~]#?cat?/etc/rsyslog.conf ........ $ModLoad?imudp $UDPServerRun?514 #?Provides?TCP?syslog?reception $ModLoad?imtcp $InputTCPServerRun?514 ......... $template?SpiceTmpl,"%msg%\n"? $template?DynaFile,"/data/external-lb/nginx/nginx-access.log" $template?SpiceTmpl2,"%msg%\n"? $template?DynaFile2,"/data/external-lb/portal/portal-access.log" ......... *.info;mail.none;authpriv.none;cron.none;local5.none;local4.none????????????????/var/log/messages ......... local5.*?????????????????????????????????????????????????DynaFile;SpiceTmpl local4.*?????????????????????????????????????????????????DynaFile2;SpiceTmpl2 重启日志接收端的rsyslog服务 [root@open-falcon01?~]#?/etc/init.d/rsyslog?restart 查看,当访问对应对应的url时,就会有转发后的文件产生,并实时有日志内容转发过来 [root@open-falcon01?~]#?ll?/data/external-lb/nginx/nginx-access.log -rw-------?1?root?root?1067372?Oct??9?10:51?/data/external-lb/nginx/nginx-access.log [root@open-falcon01?~]#?ll?/data/external-lb/portal/portal-access.log? -rw-------?1?root?root?88141?Oct??9?22:26?/data/external-lb/portal/portal-access.log

==========================================================================
注意:
a)如果发现日志还没有接收过来,即/data/rsyslog/nginx目录下没有日志产生,就同时重启推送端和接收端的rsyslog服务。确保双方的iptables防火墙和selinux关闭!
b)也可以自行修改接收的日志文件的存放路径,如改为下面的配置:
$template ?DynaFile,"/data/rsyslog/nginx/nginx-access.log"
则日志收集后存放的文件如下:
[root@log-server ~]# ll /data/rsyslog/nginx/
total 571716
-rw------- 1 root root 483539594 Jun 13 12:58 2018-06-13.log
-rw------- 1 root root 101893593 Jun 13 13:13 nginx-access.log

【本文由:湖北阿里云代理 http://www.558idc.com/aliyun.html 复制请保留原URL】
上一篇:shell脚本的条件测试与比较
下一篇:没有了
网友评论
<