Posts Tagged ‘linux’

mail command – modify the From address

Here is how you can send an email message from linux command line and use a different from address. This is useful in scripts that run as a user that would not necessarily manage the process or even look at or care about the email.

$ mail -s “Test Subject” me@mydomain.com — -f donotreply@mydomain.com -F ‘”Do Not Reply” ‘ < someinputfile

Mutt – delete messages by date range

Finally got fed up when faced with how to delete 9,000 of 10,000+ email message using mutt in linux. The key is to get the messages tagged. To tag messages, hit “T” (capital T). Then, you will be presented a “Tag messages matching:” prompt.

Tag the messages by date:

Tag messages matching: ~d dd/mm/yyyydd/mm/yyyy

For example:

Tag messages matching: ~d 01/01/2012-31/12/2012

Tagged messages will now have an “*” (asterik).

Delete all the tagged messages:
Hit the “;” (semi-colon). You will have a “tag-” prompt. Then, just hit “d”.

Configure Windows 2012 server to use NTP server.

I used the following procedure to configure my Windows 2012 server to use NTP running on my linux server.

Configure the Window Time service to use the NTP server:

PS > w32tm /config /manualpeerlist:NameOrIPAddress /syncfromflags:MANUAL

Example:

PS > w32tm /config /manualpeerlist:192.168.0.1 /syncfromflags:MANUAL

Stop the Windows Time service:

PS > Stop-Service W32Time

Start the Windows Time service:

PS > Start-Service W32Time

Query the Windows Time service:

PS > w32tm /query /status

How to use minicom to access Cisco equipment console

This is another one of those things that I do not do very often. And when it comes up, I have re-invent the wheel. No more. I will have here.

This was done from a CentOS 6.x machine connected via a Cisco console cable to a 35xx switch. This configuration has worked for any Cisco equipment I have worked with as well.

If minicom is not installed:

# yum install minicom

Setup minicom.

# minicom -s

Configure the serial port by selecting “Serial port setup” from the menu.

Select the serial port using A:
A (ttyS0)

Configure the port connection by selecting E:
E
C (9600)
V (Bits)
L (Parity)
W (Stopbits)

9600 8N1

Toggle the hardware flow control:
Hardware Flow Control (off):
F

This will save the configuration as the default configuration:

Save setup as dfl

Ensure that user accounts that need to access the console are added to the dialout group.

# vigr

dialout:x:18:jim

Verify your terminal emulation. Mine was set to xterm, and none of the menuing worked. Obviously, this would need to be done in the user environment.

# export TERM=vt100

Another useful setting is the toggle to enable logging. This is very useful, especially when messing a router or switch configuration.

From the menu (CTRL-A Z), enable capture using L.

Convert UTF-16 file to UTF-8

A lot of times, I find myself creating output files in Windows that I want to use linux to parse because I am more comfortable scripting in linux than in Windows. Most times I the output UTF-16 encoded. You can tell by using the file command. I used this to convert a file generated in Windows 2003 server, so that I could work with it in CentOS 6.x.

$ file myfilename.txt
myfilename.txt: Little-endian UTF-16 Unicode text, with CRLF line terminators

Then, to convert the file:

$ iconv -f utf-16 -t utf-8 myfilename.txt > mynewfilename.txt

Using fail2ban to block/throttle SASL authentication failures

OS: CentOS6
Repository: EPEL

I had gone back and forth on how to handle SMTP Authentication requests. Would I allow it? How would I allow it? I started off, only allowing a few subnets that I knew I would need (T-Mobile, AT&T, etc). I quickly found this to be a management nightmare, so I opened access and configured all my email clients to use SSL to encrypt the traffic. This worked great except for when others found that I had the SMTP Auth enabled and open. I started getting traffic, once in a while, just banging away trying to find an account and password to use to spam from my servers. I finally got fed up, and decided to look into a way to mitigate this behavior. It became apparent to me pretty quickly that fail2ban would do exactly what I was looking for. Basically, fail2ban looks for certain strings in a log file and adds an iptables entry to a special fail2ban chain to block the IP address for a period of time. What I like about this is that I don’t have maintain a large iptables table to block unwanted subnets. Fail2ban will watch that for me. Note: my configuration blocks both port 25 and 465 to cover both port where SMTP Authentication can occur.

Here is what I did to help with my unauthorized SASL attempts:

I installed fail2ban from the EPEL repository that I had installed but not enabled. The required dependencies are listed as well.

# yum –enablerepo=epel install fail2ban

Dependency Installed:
gamin-python.x86_64 0:0.1.10-9.el6
python-inotify.noarch 0:0.9.1-1.el6
shorewall.noarch 0:4.5.4-1.el6
shorewall-core.noarch 0:4.5.4-1.el6
tcp_wrappers.x86_64 0:7.6-57.el6

# cd /etc/fail2ban/
# cp -rp jail.conf jail.conf.orig

I disabled ssh monitoring, since I do not allow that anyway, so it was not needed.

# vi jail.conf

[ssh-iptables]

# JGZ 11/20/2012
#enabled = true
enabled = false

# JGZ 11/19/2012 – enable SASL monitoring
#enabled = false
enabled = true
filter = sasl
backend = polling
# JGZ 11/19/2012
#action = iptables[name=sasl, port=smtp, protocol=tcp]
action = iptables-multiport[name=SASL, port=”smtp,465″]
sendmail-whois[name=sasl, dest=emailaddress]
# sendmail-whois[name=sasl, dest=you@example.com]
# JGZ 11/19/2012 – configure the correct log file name.
#logpath = /var/log/mail.log
logpath = /var/log/maillog

# cd /etc/fail2ban/filter.d/
# cp -p sasl.conf sasl.conf.orig

# vi sasl.conf

# JGZ 11/19/2012 – To support my postfix output
#failregex = (?i): warning: [-._\w]+\[\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed(: [A-Za-z0-9+/]*={0,2})?$
failregex = .*postfix\/smtpd.*warning: .*\[\]:.*SASL .*authentication failure

# service fail2ban start
# chkconfig fail2ban on

With this configuration, after three authentication failures the IP address will be blocked for 5 minutes. All port access will blocked for the address. In addition, an email message will be sent to email address configured in the jail.conf file. Log output is managed via syslog and sent to the /var/log/messages file.

Update:
While installing this in CentOS 7, I discovered some new things.

In jail.local to ban forever:

bantime = -1

# fail2ban-client set jailname unbanip ipaddress

Example:

# fail2ban-client set sshd unbanip 123.123.123.123

Fail2ban has a tool that will allow you to test your regex filters:

# fail2ban-regex /path/to/logfile “someregexstuff”

How to reset MySQL root password, when forgotten..

This was bit more challenging in CentOS 7:

I couldn’t get MySQL/MariaDB to start up in safe mode. I kept getting the following no matter what options I tried:

170324 15:23:36 [ERROR] Can’t find messagefile ‘/share/mysql/errmsg.sys’
170324 15:23:36 [ERROR] Aborting

I know this is probably overkill, but I finally decided I would just pass mysqld_safe, fully qualified, all the same parameters as I noticed when I had mysql running as well as the –init-file to reset the root password I forgot:

/usr/bin/mysqld_safe –basedir=/usr –datadir=/var/lib/mysql –plugin-dir=/usr/lib64/mysql/plugin –user=mysql –init-file=/tmp/a.sql –log-error=/var/lib/mysql/hostname.err –pid-file=/var/lib/mysql/hostname.pid –skip-grant-tables

I probably just needed to basedir option.

I checked this and this all I needed in CentOS 7:

mysqld_safe –basedir=/usr –skip-grant-tables &

These are the steps used to reset the root password in MySQL running on a linux system. In my case, it was CentOS 5.6.

Login as root.

Check to see what user ID MySQL is running under. In CentOS/RedHat it is mysql.

# su – mysql

Stop MySQL if it is currently running.

# kill `cat /var/run/mysqld/mysqld.pid`

or, as root:

# service mysqld stop

Create a temporary sql script containing the following:

# vi tmp.sql
UPDATE mysql.user SET Password=PASSWORD(‘newpassword‘) WHERE User=’root’;
FLUSH PRIVILEGES;

Start up MySQL in safe mode using the initialization script you just created:

# mysqld_safe –init-file=tmp.sql &

Test the new password:

# mysql -u root -p newpassword
mysql>

As root: &

# service mysqld stop
# service mysqld start

Process used in Ubuntu 12.04.
Here is the process in Ubunutu:

# service mysql stop
# mysqld –skip-grant-tables &
# mysql -u root mysql
mysql> UPDATE user SET Password=PASSWORD(‘newpassword’) WHERE User=’root’;
mysql> FLUSH PRIVILEGES;
mysql> quit
# service mysql start

How to search and modify an LDAP record from the command line.

The following was verified on openldap versions 2.0 and 2.3:

To display all attributes associated with a particular cn:
# ldapsearch -x -b ‘distinguished name‘ ‘cn=Some CN’

Or using a wildcard:
# ldapsearch -x -b ‘distinguished name‘ ‘cn=startswith*’

Example:
# ldapsearch -x -b ‘dc=mydomain,dc=com’ ‘cn=myCN’
# ldapsearch -x -b ‘dc=mydomain,dc=com’ ‘cn=myCNstartwith*’

This information can help you to determine what you want to modify.

How to modify an attribute value for a particular cn:

#ldapmodify -x -D ‘distinguished name‘ -w password -f filename

Contents of filename:

dn: distinguished name
changetype: modify
replace: attribute
attribute: newvalue

Example:
# ldapmodify -x -D ‘cn=admin,dc=mydomain,dc=com’ -w MyPassword -f myfile

# cat myfile
dn: cn=”Lastname, Firstname”, dc=mydomain, dc=com
changetype: modify
replace: mail
mail: myaccount@mydomain.com

How to block/allow proxy access via squid proxy using ADS authentication and group membership.

The following procedures were used on RedHat Enterprise Linux 5.x.

The following previous post outlined how to get the linux server joined to domain and supporting ADS authentication. This link also contains information for RHEL6.x in the comments as well :

Howto Join An Active Directory Server using RHEL5.x/CentOS5.x and Samba 3.0.x

The key is getting the /etc/squid/squid/conf right. I have included the entire configuration used. A lot of the options are not required in all situations. I have a number of things going on here. Many of which I could clean up and make better, but I wanted to leave them in here because I know this configuration works. I actually have three proxy servers (two peers) in the configuration. The configuration forwards the requests to one of the two peers based upon the site accessed. This is probably not required for most deployments.

# cat /etc/squid/squid.conf

# Port squid listens on
http_port 3128
...
hierarchy_stoplist cgi-bin ?

acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY

acl apache rep_header Server ^Apache
broken_vary_encoding allow apache

cache_dir null /tmp

access_log /var/log/squid/access.log common

# To support ADS credentials for access.
auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp
auth_param ntlm children 10
auth_param ntlm keep_alive on

# Begin Group
# To support ADS group membership requirements.
external_acl_type WindowsGroup ttl=60 children=5 %LOGIN /usr/lib/squid/wbinfo_group.pl
# End Group

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320

acl all src 0.0.0.0/0.0.0.0
# To allow sites that do not work with NTLM. Created a list of sites that do not seem to work using NTLM
# to exclude from authentication requirement.
acl allowsites url_regex -i "/etc/squid/allowedsites.acl"

# Begin Group
# ACLs to get ADS group membership. I have two groups here. One group has unrestricted access
# and the other limited. The ADS groups used here are ProxyAccess and ProxyAccessLimited.
acl InWindowsGroup external WindowsGroup ProxyAccess
acl InLimitedWindowsGroup external WindowsGroup ProxyAccessLimited
# End Group

# Begin Facebook test.
# These ACLs are set up to manage which users can access Facebook in this instance.
# List of users.
acl UnRestrictedUsers proxy_auth "/etc/squid/UnRestrictedUsers.acl"
# List of sites allowed to these users.
acl ExcemptedSites url_regex -i "/etc/squid/ExcemptedSites.acl"
# List of managed domains.
acl RestrictedDomains dstdomain "/etc/squid/RestrictedDomains.acl"
# End Facebook test.

# Begin limited access test.
# This is where the specific restrictions are made for the ProxyAccessLimited users.
acl LimitAllowedSites url_regex -i "/etc/squid/LimitAllowedSites.acl"
acl LimitRestrictions dstdomain "/etc/squid/LimitRestrictions.acl"
# End store access test.

acl MyACL proxy_auth REQUIRED
acl java browser java
acl java browser Java

acl broken dstdomain .aa.com
header_access Accept-Encoding deny broken

http_access allow all java
http_access allow all allowsites

# Begin Group
# This will allow users in the ProxyAccessLimited ADS group access to the sites listed in the
# /etc/squid/LimitAllowedSites.acl file, but access to the sites listed in the /etc/squid/LimitRestrictions.acl
# file. Note: a "." in the LimitRestrictions.acl file will limit the users only to the sites listed in the
# LimitAllowedSites.acl file.
http_access allow InLimitedWindowsGroup LimitAllowedSites !LimitRestrictions
# This will allow users in the ProxyAccess ADS group access to any sites, and permits those listed in the
# /etc/squid/UnRestrictedUsers.acl file access to the site listed in the /etc/squid/ExcemptedSites.acl file.
http_access allow InWindowsGroup UnRestrictedUsers ExcemptedSites
# This will allow users in the ProxyAccess ADS group access to any sites, except those listed in the
# /etc/squid/ExcemptedSites.acl file.
http_access allow InWindowsGroup !ExcemptedSites
# End Group

http_access deny all
never_direct allow all
# This is where the two peers are defined.
cache_peer myproxy01.domain.com parent 80 0 no-query default
cache_peer myproxy02.domain.com parent 80 0 no-query default

# This determines which peer is going to used based the sites listed in the /etc/squid/RestrictedDomains.acl
# file.
cache_peer_access myproxy01.domain.com deny RestrictedDomains
cache_peer_access myproxy02.domain.com allow RestrictedDomains

acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access deny all

http_reply_access allow all

icp_access allow all

cache_effective_group squid

coredump_dir /var/spool/squid

memory_pools off

half_closed_clients off

The .acl files listed in the configuration are either just a list of usernames (ADS or otherwise), or a list of domains. In the case of the domains, you can use the following syntax to signify all host for a particular domain:

.mydomain.com

Below, I have tried to strip out the other things to demonstrate all that is needed to support proxy access via ADS group membership.

Excerpts from /etc/squid/squid.conf:
auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp
auth_param ntlm children 10
auth_param ntlm keep_alive on
...
# Begin Group
external_acl_type WindowsGroup ttl=60 children=5 %LOGIN /usr/lib/squid/wbinfo_group.pl
# End Group
...
# Begin Group
acl InWindowsGroup external WindowsGroup ProxyAccess
acl InLocationsWindowsGroup external WindowsGroup ProxyAccessLocations
# End Group
...
# Begin store access test.
acl LocationAllowedSites url_regex -i "/etc/squid/LocationAllowedSites.acl"
acl LocationRestrictions dstdomain "/etc/squid/LocationRestrictions.acl"
# End store access test.
...
acl MyACL proxy_auth REQUIRED
...
# Begin Group
http_access allow InLocationsWindowsGroup LocationAllowedSites !LocationRestrictions
http_access allow InWindowsGroup
# End Group
...

Also, if you want to customize the default access denied page that users receive when they go to a site that they are not permitted to access, you can add you customizations to the /usr/share/squid/errors/English/ERR_ACCESS_DENIED file. I believe you can do a number of other creative things with error pages, but I have not looked in to that.

If you want to add/remove sites from the files, you just need to issue a “service squid reload” to activate the new settings.

To disable interval filesystem checks in linux.

When you have a large volume and you want to control when file system checks are done, you can disable interval checks by issuing the following command:

# tune2fs -i 0 /dev/sdax

I have had reboots take a long because the interval was up for a 2TB partition. I have disabled it and will do them at my own time choosing.

Update: You can do this permanently by changing the sixth field in /etc/fstab file for the volume/partition.

Return top

INFORMATION