DKIM on multihosting (opendkim, postsrsd)

March 19th, 2015 No comments

We host several thousands e-commerce entities at Shoptet.cz. Each on its own domain. A good half of them is using freemail as their main e-mail address, some are using our mailboxes and some use their own. We have to transmit emails generated by their customers and order processing for all of them and we offer basic newsletter solution too (along with advanced integration with MailChimp).

As anyone else we have been forced by Seznam.cz to introduce DKIM for mass emails. We have learned that Seznam.cz is going to build sender’s reputation on the combination of DKIM selector and domain – this means we must use different combination for every e-commerce entity. And we do not want to allow users to change this combination – if they behave like spammers, they should not have bad influence on other our users.

Solution

We use postfix and naturally we started with opendkim. The documentation is not the strongest part of the project, so after many hours we came with this setup:

/etc/opendkim.conf
...
SigningTable lua:/etc/opendkim/SigningTable.lua
KeyTable lua:/etc/opendkim/KeyTable.lua
...

SigningTable.lua was tough, giving us endless Segmentation fault until we discovered, we cannot return nil.

The global variable query passed to SigningTable contains content of the “From” header. This does not allow us to distinguish e-commerce entities. People could use their freemail address. Until recently there existed directive SenderHeaders Sender,From where different headers could have been used, but this had gone in opendkim 2.10.0 release. We have decided to put it back, but not that versatile – we just hardcoded support for header called: X-DKIM-Sender. Because of internal guts it must have @ in the content, so we pass information about our entity as selector=domain@dkim. You can see in the script below, that we will strip @dkim, and separate selector and domain using the = sign.

You can see we sign everything coming from this mail server with the same private key. Our public key is a TXT record on dkim.shoptet.cz. In order to create a unique selector for every entity we created wildcard CNAME: *._domainkey.news.shoptet.cz CNAME dkim.shoptet.cz. The final step is to modify the sending applications to include the X-DKIM-Sender header. If the user has verified their domain, we would pass this:

X-DKIM-Sender: shoptet=brablc.com@dkim

If they did not we will use this:

X-DKIM-Sender: myecommerce.cz=news.shoptet.cz

Resume

We have combined this with Postfix Sender Rewriting Scheme daemon (to deliver all bouncing messages and auto-replies to the real sender) and hardened SPF record for news.shoptet.cz.

SpamAssassin now shows in the first case DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU. In the later DKIM_VALID_AU is missing obviously.

This article was written to help others seeking help with opendkim, perhaps they will not have to read so much C code as I had to to figure this all out 😉

SiteBar development moved to GitHub

March 4th, 2014 8 comments

Almost 7 years since the last release, finally SiteBar 3.4 has been released. The source code has been moved from publicly accessible but privately hosted subversion repository to GitHub – https://github.com/brablc/sitebar .

Actually I do not expect any rush in getting contributions, but it is kind of nice to see, that something with a codebase from 2005 can still be useful for someone.

I will use this post and move here comments, that landed on other pages of this blog.

Categories: SiteBar Tags:

How to export videos from iPhoto and keep their date

January 13th, 2014 No comments

When importing files from family iPhones on MacBook Air, Google Picasa does not process rotation properly. I have to use iPhoto. However iPhoto does not export properly date of videos. In order to keep my archive on disk properly organized, I wrote myself two scripts.

I export images and videos from iPhoto using:

Field Value
Kind Original
File name Use filename
Subfolder Format Event Name

This will place them into subfolder on the disk. Now in each directory I run script iphoto-touch-mov-by-prev-image, which will touch the videos to have the same date/time as the closest image.

Afterwards I run all files through a script iphoto-copy-export-to-archive-dir which will put them into YEAR/YEAR-MONTH-DAY subfolder in my archive.

In order to process multiple subdirectories I use:

#!/bin/bash
for DIR in `find . -type d`; do
    iphoto-touch-mov-by-prev-image $DIR
    iphoto-copy-export-to-archive-dir /Volumes/Archive $DIR/*.*
done
Categories: MacOSX, Missing Answers Tags:

QNAP QAirPlay and standalone web applications not accessible on port 8081

December 30th, 2013 1 comment

Probably with firmware 4.0.2 or 4.0.5 I have lost possibility to run music station and QAirPlay and other applications which run on secure port 8081. I have tracked the problem down to the fact, that /etc/config/apache/extra/apache-ssl.conf is not included. Looking at /etc/init.d/Qthttpd.sh I have discovered, that it is because /sbin/getcfg QWEB Enable_SSL -u -d NULL returns 0 instead of expected 1. I have fixed the problem by running:

/sbin/setcfg QWEB Enable_SSL 1
/etc/init.d/Qthttpd.sh restart

Now apache listens on por 8081 again and I can run QNAP applications. About two hours of lost time by Googling a solution. Proper answer was missing. There have not been any nonstandard changes on my side, I just wanted to run everything on secure ports.

Categories: Missing Answers Tags:

After restart of Asus RT-N66U router a strange page occurs and does not go away

November 13th, 2013 2 comments

I have configured a something unrelated on my router and after restart it came with strange message:

Settings have been updated. Web page will now refresh.
Changes have been made to the IP address or port number. You will now be disconnected from RT-N66U.
To access the settings of RT-N66U, reconnect to the wireless network and use the updated IP address and port number.

What a wonderful message, it must have been created by some excellent creative mind!

However, this page would not go away. Deleting HTTP authentication would not help, deleting cookies either, certificates nop. Finally I remembered that there was some strange request to allow saving offline data. And here it is:

Screen Shot 2013-11-13 at 20.14.58

Just delete the data. It was my first touch with offline data and it did not work! Great experience, now I will never let any application to store its data.

Categories: Missing Answers Tags:

One liner to backup all MySQL databases

October 14th, 2013 No comments


echo "show databases" \
| mysql --skip-column-names \
| grep -v -P '^(performance|information)_schema$' \
| awk '{ print "mysqldump --events --routines --triggers " $1 " | bzip2 > /var/backups/" $1 ".sql.bz2" }' \
| sh

Categories: Missing Answers Tags:

Add GMail MX to Plesk from command line

October 10th, 2013 No comments

This is something every admin must need, but I have not found such script. So here you are.

#!/bin/bash

DOMAIN=$1

echo "\
10 aspmx.l.google.com
20 alt1.aspmx.l.google.com
20 alt2.aspmx.l.google.com
30 aspmx2.googlemail.com
30 aspmx3.googlemail.com
30 aspmx4.googlemail.com
30 aspmx5.googlemail.com\
" | awk '{print "/usr/local/psa/bin/dns -a '$1' -mx \"\" -mailexchanger " $1 " -priority " $2 }' | sh

Categories: Missing Answers Tags:

Plesk 11.5.30 with Watchdog 2.0.7 on Centos 6.4 – workaround for startup/shutdown problems

September 21st, 2013 1 comment

I have a brand new machine with configuration stated above, moreover I’m using Nginx as proxy server to Apache. And it comes loaded with a lot of problems.

  • Machine cannot be shutdown – with last message in the console:

    init: psa-monit main process ended, respawning

  • After restart the machine reports following problems to console:

    Starting nginx: nginx: [emerg] bind() to [2001:8d8:8ae:e500::ae:4abd]:80 failed (99: Cannot assign requested address)[FAILED]

  • httpd process gets loaded with strange settins – it does not see VirtualHosts and serves all requests from default host, although it has opened all virtual host log files

Root Cause

Is unclear to me, but probably it is caused by incorrect change of starting scripts from Plesk. The scripts moved between Centos 5 and Centos 6 from /etc/inittab to /etc/init/psa-monit.conf and /etc/init/psa-wdcollect.conf . And I believe the scripts are not correct. They execute too early and respawn does not correctly handle situation when stopping machine.

Premature execution seems to cause that Apache gets started on port 80, Nginx refuses start consequently. Restarting Nginx seems to fix the port binding (strange) but Apache still does not serve proper pages. As if the started instance did not have NameVirtualHost setup properly and is serving default VirtualHost.

Workaround

I’m not familiar with Centos init procedure good enough, and I believe the problem is inside Plesk code, not just in the init configuration. Changing the init script as shown below would solve all the problems. It will however disable automatic restart when the watchdog fails itself. But this seems acceptable for me – it is bettern than machine which cannot be restarted.

Yes I admit, the sleep is funny 😉

Categories: Missing Answers Tags:

Phabricator: Settings for LDAP authenticator and SBS Active Directory

August 21st, 2013 No comments

I’m trying Phabricator (perhaps switching from Bugzilla after 10 years) and I wanted an integration with LDAP service on the Active Directory of the Microsoft Windows Small Business Server – users should use their usernames to login.

It took me a while to find out correct parameters, the error provided by Phabricator was not very helpfull:

Argument 1 passed to PhutilAuthAdapterLDAP::readLDAPRecordAccountID() must be an array, null given, called in /usr/share/phabricator/libphutil/src/auth/PhutilAuthAdapterLDAP.php on line 111 and defined

Correct settings for Small Business Server are:

Setting Value
LDAP Hostname YOURSERVER.YOURDOMAIN.local
LDAP Port 3268
Base Distinguished Name OU=SBSUsers,OU=Users,OU=MyBusiness,DC=YOURDOMAIN,DC=local
Search Attribute mailNickname
Username Attribute mailNickname
Realname Attributes sn, givenName
Anonymous Username anonymous
Anonymous Password <empty>
ActiveDirectory Domain YOURDOMAIN.local
Categories: Missing Answers Tags:

Import CSV with UTF-8 into Excel for Mac

June 28th, 2013 3 comments

Excel for Mac 2011  does not support UTF-8 in CSV files. On Windows you can simply add Byte Order Mark (BOM) to the file and it will open correctly in Excel. This does not work on Mac. You can try importing the file with different encoding and some people declare success with UTF-16 Little Endian, but for me this simply does not work.

The best solution that I have found is a script made by Konrad Foerstner, which I have improved little bit, to support different colum delimiters.

Download the script from Github https://github.com/brablc/clit/blob/master/csv2xlsx.py, install library for writing Excel format sudo easy_install openpyxl.

Now you can covert the file from CSV to XLSX.

Categories: MacOSX, Missing Answers Tags: