My notes, things I have learned, and documentation I have created.

Boy, I tell ya, the more I learn about linux, the more I love it. Especially, yum. Something was wrong after I used yum to update a package. I didn’t have time to look into it in detail, so I just wanted to back out the change and downgrade the packages that were updated during the update and restore functionality. My initial thought was to restore a snapshot of the system from the night before, but I thought better and decided I would try to rollback the installation and downgrade the application. This is what I did:

Got a list of all the yum transactions on the system using the yum history command:

# yum history list all
Loaded plugins: fastestmirror
ID | Login user | Date and time | Action(s) | Altered
——————————————————————————-
18 | | 2017-03-24 17:32 | I, U | 8 EE
…

Then, based on the time, I was to determine the transaction ID to get more information about the transaction to verify I had the correct one using the yum history info command:

# yum history info 18
Loaded plugins: fastestmirror
Transaction ID : 18
Begin time : Fri Mar 24 17:32:42 2017
Begin rpmdb : 335:80c8ab3d529a99f5edc0570b5dbf0a9a2475ffda
End time : 17:34:11 2017 (89 seconds)
End rpmdb : 342:0f129cb344b7c87fe9b0f9b0ff74715215284aea
User :
Return-Code : Success
Command Line : update wikid-server-enterprise-4.2.0.b2007-1.noarch.rpm
Transaction performed with:
Installed rpm-4.8.0-55.el6.x86_64 @base
Installed yum-3.2.29-75.el6.centos.noarch @updates
Installed yum-metadata-parser-1.1.2-16.el6.x86_64 @anaconda-CentOS-201311272149.x86_64/6.5
Installed yum-plugin-fastestmirror-1.1.30-37.el6.noarch @base
Packages Altered:
Dep-Install audit-libs-python-2.4.5-3.el6.x86_64 @base
Dep-Install libcgroup-0.40.rc1-18.el6_8.x86_64 @updates
Dep-Install libsemanage-python-2.0.43-5.1.el6.x86_64 @base
Dep-Install policycoreutils-python-2.0.83-30.1.el6_8.x86_64 @updates
Dep-Install rsync-3.0.6-12.el6.x86_64 @base
Dep-Install setools-libs-3.3.7-4.el6.x86_64 @base
Dep-Install setools-libs-python-3.3.7-4.el6.x86_64 @base
Updated wikid-server-enterprise-4.2.0.b1977-1.noarch @/wikid-server-enterprise-4.2.0.b1977-1.noarch
Update 4.2.0.b2007-1.noarch @/wikid-server-enterprise-4.2.0.b2007-1.noarch
Scriptlet output:
1 Stopping Tomcat server … Success!
2 Stopping TimeCop service … Success!
3 Stopping wAuth protocol daemon … Success!
4 RADIUS protocol daemon already stopped.
5 LDAP protocol not enabled.
6 Stopping Logger service … Success!
7 Stopping database … Success!
history info

And then, to downgrade the packages, I used the yum history undo command:

# yum history undo 18
Loaded plugins: fastestmirror
Undoing transaction 18, from Fri Mar 24 17:32:42 2017
Dep-Install audit-libs-python-2.4.5-3.el6.x86_64 @base
Dep-Install libcgroup-0.40.rc1-18.el6_8.x86_64 @updates
Dep-Install libsemanage-python-2.0.43-5.1.el6.x86_64 @base
Dep-Install policycoreutils-python-2.0.83-30.1.el6_8.x86_64 @updates
Dep-Install rsync-3.0.6-12.el6.x86_64 @base
Dep-Install setools-libs-3.3.7-4.el6.x86_64 @base
Dep-Install setools-libs-python-3.3.7-4.el6.x86_64 @base
Updated wikid-server-enterprise-4.2.0.b1977-1.noarch @/wikid-server-enterprise-4.2.0.b1977-1.noarch
Update 4.2.0.b2007-1.noarch @/wikid-server-enterprise-4.2.0.b2007-1.noarch
Loading mirror speeds from cached hostfile
* base: mirror.keystealth.org
* extras: mirror.linuxfix.com
* updates: mirror.sigmanet.com
Failed to downgrade: wikid-server-enterprise-4.2.0.b1977-1.noarch
Resolving Dependencies
–> Running transaction check
—> Package audit-libs-python.x86_64 0:2.4.5-3.el6 will be erased
—> Package libcgroup.x86_64 0:0.40.rc1-18.el6_8 will be erased
—> Package libsemanage-python.x86_64 0:2.0.43-5.1.el6 will be erased
—> Package policycoreutils-python.x86_64 0:2.0.83-30.1.el6_8 will be erased
–> Processing Dependency: policycoreutils-python for package: wikid-server-enterprise-4.2.0.b2007-1.noarch
—> Package rsync.x86_64 0:3.0.6-12.el6 will be erased
—> Package setools-libs.x86_64 0:3.3.7-4.el6 will be erased
—> Package setools-libs-python.x86_64 0:3.3.7-4.el6 will be erased
–> Running transaction check
—> Package wikid-server-enterprise.noarch 0:4.2.0.b2007-1 will be erased
–> Finished Dependency Resolution

Dependencies Resolved

===
Package Arch Version Repository Size
===
Removing:
audit-libs-python x86_64 2.4.5-3.el6 @base 279 k
libcgroup x86_64 0.40.rc1-18.el6_8 @updates 331 k
libsemanage-python x86_64 2.0.43-5.1.el6 @base 312 k
policycoreutils-python x86_64 2.0.83-30.1.el6_8 @updates 1.3 M
rsync x86_64 3.0.6-12.el6 @base 682 k
setools-libs x86_64 3.3.7-4.el6 @base 1.1 M
setools-libs-python x86_64 3.3.7-4.el6 @base 1.6 M
Removing for dependencies:
wikid-server-enterprise noarch 4.2.0.b2007-1 @/wikid-server-enterprise-4.2.0.b2007-1.noarch 99 M

Transaction Summary
===
Remove 8 Package(s)

Installed size: 104 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Stopping Tomcat server … Success!
TimeCop process already stopped.
wAuth protocol daemon already stopped.
RADIUS protocol daemon already stopped.
LDAP protocol not enabled.
Stopping Logger service … Success!
Stopping database … Success!
Erasing : wikid-server-enterprise-4.2.0.b2007-1.noarch 1/8
Erasing : policycoreutils-python-2.0.83-30.1.el6_8.x86_64 2/8
Erasing : setools-libs-python-3.3.7-4.el6.x86_64 3/8
Erasing : setools-libs-3.3.7-4.el6.x86_64 4/8
Erasing : audit-libs-python-2.4.5-3.el6.x86_64 5/8
Erasing : libcgroup-0.40.rc1-18.el6_8.x86_64 6/8
Erasing : libsemanage-python-2.0.43-5.1.el6.x86_64 7/8
Erasing : rsync-3.0.6-12.el6.x86_64 8/8
Verifying : rsync-3.0.6-12.el6.x86_64 1/8
Verifying : wikid-server-enterprise-4.2.0.b2007-1.noarch 2/8
Verifying : policycoreutils-python-2.0.83-30.1.el6_8.x86_64 3/8
Verifying : libsemanage-python-2.0.43-5.1.el6.x86_64 4/8
Verifying : setools-libs-python-3.3.7-4.el6.x86_64 5/8
Verifying : libcgroup-0.40.rc1-18.el6_8.x86_64 6/8
Verifying : audit-libs-python-2.4.5-3.el6.x86_64 7/8
Verifying : setools-libs-3.3.7-4.el6.x86_64 8/8

Removed:
audit-libs-python.x86_64 0:2.4.5-3.el6 libcgroup.x86_64 0:0.40.rc1-18.el6_8 libsemanage-python.x86_64 0:2.0.43-5.1.el6 policycoreutils-python.x86_64 0:2.0.83-30.1.el6_8
rsync.x86_64 0:3.0.6-12.el6 setools-libs.x86_64 0:3.3.7-4.el6 setools-libs-python.x86_64 0:3.3.7-4.el6

Dependency Removed:
wikid-server-enterprise.noarch 0:4.2.0.b2007-1

Complete!

Now, in my case, I was not able downgrade the software package directly, as you can tell from the “Failed to downgrade:” message for the wikid server, the heavy lifting was done.
All I had to do, is install the original package using yum, and I was back in business:

# yum install wikid-server-enterprise-4.2.0.b1977-1.noarch.rpm

And, start the application back up:

# wikidctl start

CentOS: 7.x
Samba: 4.4.4

Just a quick note. If you want to list the users in your smbpasswd file, you can use the pdbedit command.

# pdbedit -L
No builtin backend found, trying to load plugin
Module ‘tdbsam’ loaded
username:1002:

Or, for more detailed output:

# pdbedit -L -v
No builtin backend found, trying to load plugin
Module ‘tdbsam’ loaded
—————
Unix username: username1
NT username:
Account Flags: [U ]
User SID: S-1-5-21-856554280-4097225363-552893113-1000
Forcing Primary Group to ‘Domain Users’ for username1
Primary Group SID: S-1-5-21-856554280-4097225363-552893113-513
Full Name:
Home Directory: \\server\username1
HomeDir Drive:
Logon Script:
Profile Path: \\server\username1\profile
Domain: SERVER
Account desc:
Workstations:
Munged dial:
Logon time: 0
Logoff time: Wed, 06 Feb 2036 07:06:39 PST
Kickoff time: Wed, 06 Feb 2036 07:06:39 PST
Password last set: Thu, 19 Jan 2017 15:10:20 PST
Password can change: Thu, 19 Jan 2017 15:10:20 PST
Password must change: never
Last bad password : 0
Bad password count : 0
Logon hours : FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
—————
Unix username: username2
NT username:
Account Flags: [U ]
User SID: S-1-5-21-856554280-4097225363-552893113-1001
Forcing Primary Group to ‘Domain Users’ for username2
Primary Group SID: S-1-5-21-856554280-4097225363-552893113-513
Full Name:
Home Directory: \\server\username2
HomeDir Drive:
Logon Script:
Profile Path: \\server\username2\profile
Domain: SERVER
Account desc:
Workstations:
Munged dial:
Logon time: 0
Logoff time: Wed, 06 Feb 2036 07:06:39 PST
Kickoff time: Wed, 06 Feb 2036 07:06:39 PST
Password last set: Mon, 20 Mar 2017 16:08:57 PDT
Password can change: Mon, 20 Mar 2017 16:08:57 PDT
Password must change: never
Last bad password : 0
Bad password count : 0
Logon hours : FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

You can add users to the smbpasswd file as follows:

# smbpasswd -a username3

To set a forwarding address for a mailbox that will also deliver the message to the Identity’s mailbox:

Set-Mailbox -Identity identityname -DeliverToMailboxAndForward $true -ForwardingSMTPAddress “my@mail.address“

Note: If DeliverToMailboxAndForward is $false, then the message will not be deliver to the Identity’s mailbox. Just forwarded.

To remove a forwarding address for a mailbox:

Set-Mailbox -Identity identityname -ForwardingSMTPAddress $null -ForwardingAddress $null

To delete email from a mailbox:
Search-Mailbox -Identity someuser -SearchQuery ‘Received:someuser -SearchQuery ‘Sent:Get-MailboxAutoReplyConfiguration accountname | Format-List

To disable auto reply for an account:

Set-MailboxAutoReplyConfiguration accountname -AutoReplyState:disabled

To add an email address to a mailbox:

Set-Mailbox accountname -EmailAddresses @{add=”email@address.domain“}

To only restore the Inbox for a mailbox:

Restore-Mailbox -Identity restoreto -RecoveryDatabase recoverydbname -RecoveryMailbox restorefrom -Target targetfolder -IncludeFolders \Inbox

To disable a mailbox means to disconnect a mailbox from an account. It doesn’t remove either the account or the mailbox. All Exchange attributes will be removed from the account and the mailbox will be in a disabled state. Note: The mailbox does not immediatley show up in the disabled mailbox list until the Clean-MailboxDatabase process runs again. This periodically happens on the system, but can be forced with the Clean-MailboxDatabase command. To disable a mailbox:

Disable-Mailbox -Identity “Display Name“

Update list of disabled mailboxes:

Clean-MailboxDatabase “Mailbox Database Name“

To list all disabled mailboxes:

Get-MailboxDatabase | Get-MailboxStatistics | Where { $_.DisconnectReason -eq “Disabled” }

To connect a mailbox to a user account:

Connect-Mailbox -Database DatabaseName -Identity “Mailbox Display Name” -User accountname

Note: For the Identity or User, you can use the Legacy DN. I found this useful, when I had two of the same Display Name mailboxes disabled at the same time. You can determine the Legacy DN using the following:

Get-MailboxDatabase | Get-MailboxStatistics | Where { $_.DisconnectReason -eq “Disabled” | Select DisplayName,LegacyDN}

To check you send filter configuration:

Get-SenderFilterConfig

Note: This will override the current setting and only block the one address.
To block a user:

Set-SenderFilterConfig -BlockedSenders emailaddress

To block more than one:

Set-SenderFilterConfig -BlockedSenders emailaddress1,emailaddress2

To block an entire domain:

Set-SenderFilterConfig -BlockedDomains domainname

The BlockedSenders and BlockedDomains are “Multivalue properties”, so to add or remove entries instead of entering all them every time you just want to add one, you can do the following:

Set-SenderFilterConfig -BlockedSenders @{Add=”emailaddress1“, “emailaddress2“}

Or:

Set-SenderFilterConfig -BlockedDomains @Add={“domainname1“, “domainname2“

Same idea to remove from the list:

Set-SenderFilterConfig -BlockedSenders @{Remove=”emailaddress1“, “emailaddress2“}

Or:

Set-SenderFilterConfig -BlockedDomains @Remove={“domainname1“, “domainname2“

Managing message size:
Here are the places where sending and receiving message size can be managed:

Get-TransportConfig | FL

Get-ReceiveConnector | FL

Get-SendConnector | FL

Get-Mailbox mailbox | FL

You want too look at the MaxSendSize, MaxReceiveSize settings.

Get-TransportConfig | FT MaxSendSize, MaxReceiveSize

Get-ReceiveConnector | FT MaxMessageSize

Get-SendConnector | FT Name,MaxMessageSize

Get-Mailbox mailbox | FT Name,MaxSendSize, MaxReceiveSize

get-transportconfig | Set-TransportConfig -maxsendsize 15MB -maxreceivesize 15MB; get-receiveconnector | set-receiveconnector -maxmessagesize 10MB; get-sendconnector | set-sendconnector -maxmessagesize 10MB; get-mailbox | Set-Mailbox -Maxsendsize 10MB -maxreceivesize 10MB

To modify the limits:

Set-TransportConfig -MaxSendSize 100MB -MaxReceiveSize 100MB

Set-ReceiveConnector -MaxMessageSize 100MB

Set-SendConnector connectorname -MaxMessageSize 100MB

Get-Mailbox mailbox | Set-Mailbox -MaxSendSize 100MB -MaxReceiveSize=100MB

There is a way to limit the attachment size itself, but creating a transport rule:

New-TransportRule -Name GTHMaxAttachSize -AttachmentSizeOver 100MB -RejectMessageReasonText “This attachment is too big! What were you thinking?”

Distribution Groups:
Create a new group:
New-DistributionGroup -Name “group name” -Alias “groupalias” -OrganizationalUnit ‘oudn‘

Modify a group setting:
Set-DistributionGroup groupalias -RequireSenderAuthenticationEnabled $false

Update a group member:
Update-DistributionGroupMember groupalias -Members “emailaddress”

Add a group member:
Add-DistributionGroupMember groupalias -Member “emailaddress”

Remove a group member:
Remove-DistributionGroupMember groupalias -Member “emailaddress”

Look at at distribution group:
Get-DistributionGroup groupalias | Format-List

Get the list of members in a distribution group:
Get-DistributionGroupMember groupalias | Format-Table

Check deleted item policy on a mailbox:
Get-Mailbox alias | Select Name,RetainDeletedItemsFor,RetainDeletedItemsUntilBackup

How to find mailboxes that are hidden:
Get-Mailbox | where {$_.HiddenFromAddressListsEnabled -eq $true}

This is a good command to list all options/members of an object. In this case a mailbox. Get-Member can used like this against really any EMS command:

Get-Mailbox | Get-Member

To restore the deleted items, right mouse click on the “Deleted Items” folder and select “Recover Deleted Items.” A window will come up with all the messages still available within the deleted item retention policy.

To see if a mailbox has auditing enabled:

Get-Mailbox mailboxname | Select Name,AudtiEnabled

To enable auditing on a mailbox:

Set-Mailbox -Identity “mailboxidentity” -AuditEnabled $true

To search the audit log for activity in a mailbox:

Search-MailboxAuditlog -Identity “mailboxidentity“

Note: Enabling auditing seems to take a while before any results show up.

An easy way to list all the folders and size in a mailbox:
Get-MailboxFolderStatistics -identity “myidentity” | select FolderPath,FolderSize

Here is a very good way to search through the logs using EMS:
Get-MessageTrackingLog -Start “mm/dd/yyyy HH:MM” -End “mm/dd/yyyy HH:MM” -ResultSize Unlimited-Sender sender@email.address | Where {($_.Recipients -like “*somestring*”)} | Select Sender,Recipients,TimeStamp,MessageSubject

CentOS 7
VMware ESXi

I used the following method to remove a disk from a system. Note: In this case the volume group only had one volume on it.

Disk: /dev/sde
Volume Name: /dev/vg_name/lv_name
Volume Group Name: vg_name

Remove all volumes in a group:

# lvremove vg_name
Do you really want to remove active logical volume lv_name? [y/n]: y
Logical volume “lv_name” successfully removed

Remove the group:

# vgremove vg_name
Volume group “vg_name” successfully removed

Remove the volume:

# pvremove -v /dev/sde
Wiping cache of LVM-capable devices
Labels on physical volume “/dev/sde” successfully wiped

I deleted the disk from vSphere, and then rescan for the changes:

# echo “- – -” > /sys/class/scsi_host/host0/scan

Verify that the disk is no longer there:

# fdisk -l /dev/sde

I decided to give OpenVAS as an alternative to Nessus thinking it would be pretty comparable since it is Nessus fork and the Nessus cost was too much for a small company. I used the following to install it on CentOS7.

First the requirements. They are not clearly defined on the OpenVAS page for downloading the binary packages, which I did. You need to disable SElinux. I had mine in permissive mode and it caused some problems. The rest was pretty straight forward. This uses the Atomicorp repository.

To install and perform initial configuration:

# wget -q -O – http://www.atomicorp.com/installers/atomic |sh
# yum upgrade
# yum install openvas
# openvas-setup

To stop, start and check OpenVAS services:

# systemctl stop openvas-manager
# systemctl status openvas-manager
# systemctl start openvas-manager
# systemctl status openvas-scanner
# systemctl stop openvas-scanner
# systemctl start openvas-scanner

Location of the logs:

# cd /var/log/openvas/
# tail gsad.log
# tail openvassd.log
# tail openvasmd.log

This is a very useful command to very the status of you installation. It was helpful in determining that I needed to disable SElinux:

# openvas-check-setup

This command rebuilds the database information:

# openvasmd –rebuild

As a result of not having SElinux disabled, I found that the redis (an advanced key-value store) service was not running so the OpenVAS scanner would not work properly after I rebooted. With SElinux disabled, I restarted redis.
# systemctl stop redis
# systemctl start redis
# systemctl status redis

And then to check the status:

# openvas-check-setup
openvas-check-setup 2.3.7
Test completeness and readiness of OpenVAS-8
(add ‘–v6’ or ‘–v7’ or ‘–v9’
if you want to check for another OpenVAS version)

Please report us any non-detected problems and
help us to improve this check routine:
http://lists.wald.intevation.org/mailman/listinfo/openvas-discuss

Send us the log-file (/tmp/openvas-check-setup.log) to help analyze the problem.

Use the parameter –server to skip checks for client tools
like GSD and OpenVAS-CLI.

Step 1: Checking OpenVAS Scanner …
OK: OpenVAS Scanner is present in version 5.0.7.
OK: OpenVAS Scanner CA Certificate is present as /var/lib/openvas/CA/cacert.pem.
OK: redis-server is present in version v=3.0.7.
OK: scanner (kb_location setting) is configured properly using the redis-server socket: /tmp/redis.sock
OK: redis-server is running and listening on socket: /tmp/redis.sock.
OK: redis-server configuration is OK and redis-server is running.
OK: NVT collection in /var/lib/openvas/plugins contains 51943 NVTs.
WARNING: Signature checking of NVTs is not enabled in OpenVAS Scanner.
SUGGEST: Enable signature checking (see http://www.openvas.org/trusted-nvts.html).
OK: The NVT cache in /var/cache/openvas contains 51943 files for 51943 NVTs.
Step 2: Checking OpenVAS Manager …
OK: OpenVAS Manager is present in version 6.0.9.
OK: OpenVAS Manager client certificate is present as /var/lib/openvas/CA/clientcert.pem.
OK: OpenVAS Manager database found in /var/lib/openvas/mgr/tasks.db.
OK: Access rights for the OpenVAS Manager database are correct.
OK: sqlite3 found, extended checks of the OpenVAS Manager installation enabled.
OK: OpenVAS Manager database is at revision 146.
OK: OpenVAS Manager expects database at revision 146.
OK: Database schema is up to date.
OK: OpenVAS Manager database contains information about 51943 NVTs.
OK: At least one user exists.
OK: OpenVAS SCAP database found in /var/lib/openvas/scap-data/scap.db.
OK: OpenVAS CERT database found in /var/lib/openvas/cert-data/cert.db.
OK: xsltproc found.
Step 3: Checking user configuration …
WARNING: Your password policy is empty.
SUGGEST: Edit the /etc/openvas/pwpolicy.conf file to set a password policy.
Step 4: Checking Greenbone Security Assistant (GSA) …
OK: Greenbone Security Assistant is present in version 6.0.11.
Step 5: Checking OpenVAS CLI …
OK: OpenVAS CLI version 1.4.5.
Step 6: Checking Greenbone Security Desktop (GSD) …
SKIP: Skipping check for Greenbone Security Desktop.
Step 7: Checking if OpenVAS services are up and running …
OK: netstat found, extended checks of the OpenVAS services enabled.
OK: OpenVAS Scanner is running and listening on all interfaces.
OK: OpenVAS Scanner is listening on port 9391, which is the default port.
OK: OpenVAS Manager is running and listening on all interfaces.
OK: OpenVAS Manager is listening on port 9390, which is the default port.
OK: Greenbone Security Assistant is listening on port 80, which is the default port.
Step 8: Checking nmap installation …
WARNING: Your version of nmap is not fully supported: 6.47
SUGGEST: You should install nmap 5.51 if you plan to use the nmap NSE NVTs.
Step 10: Checking presence of optional tools …
OK: pdflatex found.
WARNING: PDF generation failed, most likely due to missing LaTeX packages. The PDF report format will not work.
SUGGEST: Install required LaTeX packages.
OK: ssh-keygen found, LSC credential generation for GNU/Linux targets is likely to work.
OK: rpm found, LSC credential package generation for RPM based targets is likely to work.
WARNING: Could not find alien binary, LSC credential package generation for DEB based targets will not work.
SUGGEST: Install alien.
OK: nsis found, LSC credential package generation for Microsoft Windows targets is likely to work.
OK: SELinux is disabled.

It seems like your OpenVAS-8 installation is OK.

If you think it is not OK, please report your observation
and help us to improve this check routine:
http://lists.wald.intevation.org/mailman/listinfo/openvas-discuss
Please attach the log-file (/tmp/openvas-check-setup.log) to help us analyze the problem.

To update the rules or tests (Network Vulnerability Tests – NVTs), you can use the following command which is run by the setup when you run it. I believe this is the one that very clearly says that you should at most run this once a day, otherwise they will block you IP address. It looks like they usually updated once a week anyway. If so, you will get something like the following:

# openvas-nvt-sync
[i] This script synchronizes an NVT collection with the ‘OpenVAS NVT Feed’.
[i] The ‘OpenVAS NVT Feed’ is provided by ‘The OpenVAS Project’.
[i] Online information about this feed: ‘http://www.openvas.org/openvas-nvt-feed.html’.
[i] NVT dir: /var/lib/openvas/plugins
OpenVAS community feed server – http://www.openvas.org/
This service is hosted by Greenbone Networks – http://www.greenbone.net/

All transactions are logged.

If you have any questions, please use the OpenVAS mailing lists
or the OpenVAS IRC chat. See http://www.openvas.org/ for details.

By using this service you agree to our terms and conditions.

Only one sync per time, otherwise the source ip will be blocked.

[i] Feed is already current, no synchronization necessary.

I have been using Nagios Log Server, and seem to be running across various issues with it. Most of it seems to be related to running out disk space where the elasticsearch indexes are stored. I highly recommend that you do not allow that to happen. Clear and concise documentation is sketchy for this flexible and powerful centralized log server. I have decided to post a few notes of things I have stumbled onto that help me to be able manage the process better.

I created several alerts, and I could get them to work by manually having the query executed in the Alerting tab. However, they did not seem to firing off at the Check Interval I had specified. None of them seem to be. Upon trying to resolve this issue, I discovered a couple troubleshooting tips to note for future reference. Note: In my case, none these revealed the cause of my issue. At lease, I don’t think they did. Nonetheless, here they are.

Check the poller:

[nagios]$ /usr/bin/php /var/www/html/nagioslogserver/www/index.php poller
Updating Cluster Hosts File
Updating Elasticsearch with instance…
Updating Cluster Hosts File
Updating Elasticsearch with instance…
Updating Cluster Hosts File
Updating Elasticsearch with instance…
Updating Cluster Hosts File
Updating Elasticsearch with instance…
Finished Polling.

Check the jobs:

[nagios]$ /usr/bin/php /var/www/html/nagioslogserver/www/index.php jobs
Processed 0 node jobs.
Processed 0 global jobs.

Look for an error in the cron log:

# grep ERROR /var/log/cron

What fixed my issue was going into the Administration tab and selecting “Command Subsystem” on the left side. From there, clicking “Reset All Jobs” resolved my issue.

Also, in the Administration tab, if you select “Audit Reports” you can some verification that the alerts are running. Before, resetting all the jobs, it was clear that were not running. After reseting, I see several regular scheduled entries regarding the returned messages from the alerts.

—

Another thing I was able to put together was report. In particular, I was looking to create a daily report of all IP addresses that made an attempt to login to one an externally facing server. I did this by creating an elasticsearch alert query, and then tweaking the time. I discovered that I could copy the query and execute it in a shell script using the curl command. Now, I had found references to people doing this, but they were using a curl switch of -XGET. This never worked for me, but -XPOST did and has been for a quite a while. Once you have the query copied from the dashboard query, you just need to past into a file and change the -XGET to -XPOST. Make the file executable, and then run it to get the text output. I wrapped some bash code around the query and formatted the output to create a report. Could be very useful.

—

This one really frustrated me and I am still not sure I am doing it right, but it seems to be working. As I said earlier, I kept running out of space. My index space was too large, so I just wanted to purge/delete the old ones to conserve space. Nothing in the UI seemed to work. The repositories in the Administration tab under “Backup & Maintenance” seem finicky and sensitive at best. Again, not really easy to find, I discovered some information about the curator command for elasticsearch. I used this with some parameters to effectively manage my index retention. I run these commands as user nagios as nightly job in cron:

To create a snapshot and save to your backup repository:

curator snapshot –repository nameofbackuprepository indices –older-than numberofdaystokeep –time-unit days –timestring %Y.%m.%d

To close the indices:

curator close indices –older-than numberofdaystokeep –time-unit days –timestring %Y.%m.%d

To delete the indices:

curator delete indices –older-than numberofdaystokeep –time-unit days –timestring %Y.%m.%d

To list your repositories via command line:

curl -XGET “localhost:9200/_snapshot?pretty”

To force the backup to run and create a snapshot:

curator snapshot –repository “RepositoryName” indices –all-indices

There are switches to curator command that you can use to get more verbose output and send that output to a log file:

Verbosity:

–loglevel level

Level options available, found on the Elastic site (https://www.elastic.co/guide/en/elasticsearch/client/curator/current/configfile.html):

CRITICAL will only display critical messages.
ERROR will only display error and critical messages.
WARN will display error, warning, and critical messages.
INFO will display informational, error, warning, and critical messages.
DEBUG will display debug messages, in addition to all of the above.

Capture output:

–logfile /tmp/test_backup.txt