|
|
|
|
|
|
|
|
DynaStop is a gpl licensed
LINUX utility to examine IP4 based addresses for Exim and
procmail for the purpose of filtering based upon patternistic
rules defined by the administrator.
DynaStop will work with any mail transfer agent/utility
(like DSpam) that can run external programs
or inline filters (piped I/O) and respond to error codes. I would be most appreciative
to anyone doing so and would like to share the medthod of incorporating DynaStop into
their mail system. Full Credit and a link will be gladly provided.
This can be a pivotal factor in e-mail filtering and server load management
as dynamic IP addresses are typically used for dial-up, dhcp, and DSL accounts.
All of which have a designated mail exchange server that all outbound mail flows
as defined with many if not most large Internet Service Providers (ISP) such as Road Runner,
ATT, Qwest, PacBell, BellSouth, EarthLink, AOL, and many more around the world
if their terms of service or acceptable use policy.
With DynaStop, an Exim
administrator or procmail administrator/user can filter
incoming mail on the basis of whether it is a dynamic IP address, thus reducing system load and
resources in processing unwanted mail as all legitimate mail from any given ISP will be sent from
a proper mail exchange server.
A discussion form is available at
exim-users.org. Many thanks to
Kevin Barrett for hosting the forum.
Many thanks to Tom Knaupp for contributions in
DynaStop's config files. Tom has provided a list of false positives. His list is in the DynaStop.TomKnaupp.conf
file.
Many thanks to Mike Cardwell for developing and testing a macro for Exim
that allows it comminucate with the DynaWall server without the use of the
DynaStop module and for suggesting the the web update script should be
included in the distribution file. The section for the Cardwell Macro is here.
Please visit DynaStop's Freashmeat
page, and rate it.
DynaStop requirements:
Files available for download:
| File | Description | Last Updated |
| DynaStop vers.11023.1657.1368956396 | Source code for DynaStop/Wall. Includes all the below files. | 2013-05-19 04:41:43 |
| DynaStop.AutoGreen.conf | Examples of DynaStops auto green listing abilities for systems that pass the grey listing timeouts | 2010-07-21 15:20:06 |
| DynaStop.BleedingThreats.conf | The EmergingThreats List | 2013-05-18 04:40:56 |
| DynaStop.Bogons.conf | The CYMRU Bogons List | 2012-04-28 04:48:56 |
| DynaStop.StopIP.conf | The Stop-IP Blacklist | 2013-04-07 04:45:34 |
| DynaStop.Confirmed.conf | Confirmed IP addresses: Absolutely NO false positives as stated by the IP address owner | 2009-08-29 04:43:13 |
| DynaStop.DropList.conf | The Spamhaus Drop List | 2013-05-19 04:39:45 |
| DynaStop.Excludes.conf | Examples of domains that should never be evaluated by DynaStop | 2010-07-21 15:20:06 |
| DynaStop.Expressions.conf | Examples of ways to create macros of common patterns for easy re-use | 2010-11-20 01:12:18 |
| DynaStop.NoFilter.conf | This example demonstrates how receipent domains can receive unfiltered mail in a large system hosting hundreds or thousands of domains | 2009-08-29 04:43:13 |
| DynaStop.NoMail.conf | Example of places you may never want e-mail from | 2010-07-21 15:20:06 |
| DynaStop.Patterns.conf | Large list of dynamic IP address patterns and the heart of the system | 2012-12-19 04:52:10 |
| DynaStop.RBL.conf | Lists multiple real time black lists and demonstrates how their usage is combined by DynaStop to lower false positives | 2012-03-09 18:03:20 |
| DynaStop.Subject.conf | Example subject lines taken from spam my server has received | 2009-08-29 04:43:13 |
| DynaStop.TomKnaupp.conf | Tom's Exclusion List | 2009-08-29 04:43:13 |
| DynaStop.Virus.conf | The Virus List: IP addresses of infected computers | 2013-05-19 04:39:56 |
| DynaStop.WebHelper4U.conf | Any site directly or indirectly associated with the creation, distribution, and or promotion of any type of electronic form of vulnerability and or malware as defined by the legitimate security community that infects hardware and or software. | 2009-08-29 04:43:15 |
| DynaStop.WhiteList.conf | The DNS WhiteList | 2012-10-21 04:41:19 |
How DynaStop was tested and the research involved (The research data is NOT required to run DynaStop):
The research used to develop DynaStop was started in 1999 using a heuristic approach to analyze reverse IP address schemes. Use this only for analytical and research purposes. This process is VERY CPU intensive and not recommended for a gateway server to do (though an internet connection is required for the DNS lookups). The test takes several weeks to complete on a 1.7GHz AMD. You may wish to redirect the output to a file.
Over 373 Million IP addresses were used in testing DynaStop for integrity and stability in identifying dynamic addresses and cross checked false positives.
Click here for comparision and verification tables of a given data set.A simple script can be used to generate random IP addresses:
#!/bin/bash i=0 while [ $i -le $1 ] ; do let ip1=$RANDOM%256 let ip2=$RANDOM%256 let ip3=$RANDOM%256 let ip4=$RANDOM%256 echo $ip1.$ip2.$ip3.$ip4 let i=i+1 doneAfter DynaStop has been installed and configured and the above script has been saved as ipaddr.sh, the following command can be used as a single line command:
./ipaddr.sh 20 | /usr/exim/DynaTest /usr/exim/DynaStop.conf
The columns printed out are as follows (This applies to all DynaTest/DynaCheck output):
Return Code
IP Address
Reverse Domain Name
Wordy result of analysisMatched components after IP substitution, M:
The orignal configuration line that is to be compared, C:
or a P: and the pattern DynaTest believe will match this IP addressOf the 373,302,000 IP addresses tested (Aug 1/07) 85% (317,306,720) where evaluated to be dynamic. These numbers do not include IP addresses that do not have a reverse domain name. The estimated 10% ligitimate mail servers that use dynamic IP addresses (based upon user feedback) can easily be excluded leaving 279,976,512 IP addresses (potential spam zombies) that will be blocked/tagged by DynaStop.
The graph on the right is an illustration of this system's spam filtering from December
of 2005 to the middle of Febuary of 2006. The timeline prior to the end of January illustrates
Exim using only HELO forgery tests (at most 10% of total, averaging around 5% daily). The values
on the graph of the daily total percentage of spam identified. Starting in late January
(the 27th), DynaStop was brought online. The remainder of January to current
shows the impact of Exim with HELO forgery tests combined with DynaStop.
It should be noted that antivirus and other antispam messures are performed external
to Exim where resource management has better control. Antispam and antivirus should run in the
order of least expensive to most expensive. The expense of any given method is the cost of
bandwidth and processing power.
Below is one year of data illustrating this system antispam defenses. It is updated on a
daily basis.
Pragmatic Analysis of DynaSyop
Setting up DynaStop is a follows:
Be sure to check the COMPILE script and modify the CPU variable as needed. Below is the Compilation script used to build DynaStop (included in the download):
#!/bin/bash clear # Modify for your CPU using GCC switches export CPU="-march=i686 -mtune=i686" # Required defines export DEF="-D_USE_LARGEFILE64 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -DDynaWall" export SRC="-D_REENTRANT -D_BSD_SOURCE" # Required Libraries export LIB="-lresolv -lpcre" # Required compilier options export OPT="-w -O3 -funroll-loops -ffast-math -march=i686 -mtune=i686 -fomit-frame-pointer -fno-exceptions" # Get down to business and compile it out. for i in Stop Test ; do gcc -o Dyna$i Dyna$i.c $DEF $CPU $SRC $LIB $OPT strip Dyna$i done ### ### Special defines (Greylisting flavors, case IS important): ### ### Definations: ### RDN Reverse Domain Name ### ### GreyList Activates greylisting functions ### REQUIRED for any greylisting operations ### ### GreyEvery Greylist every NON-DYNAMIC ip address if this flag ### is present. If this flag is NOT present, then ### greylist dynaic IP addresses ### ### AutoGreen Automatically write green listed entries to the ### AutoGreen file. ### ### RDN Only reversible IP addresses are greylisted. ### IP addresses with nor reverse domain name have a ### return code 2 issued. ### Only useful when NOT using GreyEvery. ### ### Uncomment ONE of the following, default Greylisting with AutoGreen ### For NO greylisting, make sure all the SP lines have a # infront of them ### Greylisting only of DYNAMIC IP addresses with a RDN #SP="-DGreyList -DRDN" # Greylisting with AutoGreen saves SP="-DGreyList -DAutoGreen -DRDN" # Greylisting NON dynamic IP address, return codes 1/2 for dynamic IP addresses #SP="-DGreyList -DGreyEvery" # Greylisting NON dynamic IP address, return codes 1/2 for dynamic IP addresses With AutoGreen saves #SP="-DGreyList -DGreyEvery -DAutoGreen" gcc -o DynaWall.greylist DynaWall.shram.c $DEF $CPU $SRC $LIB $OPT $SP strip DynaWall.greylist
After adjusting the CPU parameters to match your hardware and selecting the compile options, simply type:
./COMPILE
If all went well, the prompt will return and no errors will be displayed.
For easy administration, I recommend coping the DynaStop, DynaWall, and DynaTest executables into your Exim base folder (procmail users can change the folder as appropriate, the change does have to be consistent, this includes the DynaStop conf files), like so:
cp DynaWall.shram /usr/exim/DynaWall
cp DynaStop /usr/exim
cp DynaTest /usr/exim
Now on to the DynaStop configuration file. Below is an example of a basic configuration. This is in no means complete or accurate to any system. It is a basic blueprint to tailor to your system's specific needs. An example is included in the download.
### DynaStop configuration file. ### ### Control verbs DynaStop uses are: ### NoFilter - Receipent domain names that do NOT want any filtering ### Exclude - Dynamic patterns that are EXCLUDED at all costs from DynaStop ### ExRange - IP ranges that are EXCLUDED at all costs from DynaStop ### ExCidr - IP CIDRs that are EXCLUDED at all costs from DynaStop ### Pattern - Defines a dynamic pattern to search for ### PatRange - Dynamic IP ranges to search for ### PatCidr - Dynamic IP ranges to search for ### NoMail - A pattern that describes a site you do NOT want to receive mail from ### NoMailRange - IP ranges that are NOT accepted at all costs from DynaStop ### NoMailCidr - IP CIDRs that are NOT accepted at all costs from DynaStop ### Include - Includes a seperate file into DynaStop. Same format as this file and requires full pathname ### ### All patterns ekements must match for the pattern to be matched ### ### Patterns that match dynamic IP addresses ### ### IP Number breakdown: ### ### IP address: 126.12.34.56 ### Reverse IP: softbank126012034056.bbtec.net ### ### $1 - $4 are the decimal IP address parts ### $5 - $8 are the hex IP address parts ### ### $1 = 126 $5 = 7E ### $2 = 12 $6 = 0C ### $3 - 34 $7 = 22 ### $4 = 56 $8 = 38 ### ### See also: ### http://tools.ietf.org/id/draft-msullivan-dnsop-generic-naming-schemes-00.txt ### ### WARNING: This is is based completely on heuristic analysis. ### WARNING: BE SURE YOU EXAMINE IT AND YOUR LOGS VERY FREQUENTLY ### WARNING: TO PREVENT UNWANTED BLOCKING if you use DENY or DROP ### WARNING: in your Exim configuration. USE AT YOUR OWN RISK. ### WARNING: THE CONTENTS IN THIS FILE ARE FOR EXAMPLE ONLY. ### ### WARNING: BE SURE YOU EXCLUDE YOUR OWN LOCAL NETWORKS. ### ### GreyList/Cache timeouts ### ### MaxTTL - Maximum time in HOURS an item remains in the cache MaxTTL 24 ### GreyTTL - Minimium time in MINUTES an IP address has to wait ### to be GreenListed. Ignored if GreyList not enabled GreyTTL 1 ### ### File to write AutoGreen entries. Ignored if AutoGreen ### is not enabled ### AutoGreen /usr/exim/DynaStop.AutoGreen.conf ### ### Expressions must always be first ### Include /usr/exim/DynaStop.Expressions.conf ### ### Load the NoFilter patterns ### Include /usr/exim/DynaStop.NoFilter.conf ### ### Load the Dynamic IP patterns, Excludes, and NoMail sections ### Include /usr/exim/DynaStop.Confirmed.conf Include /usr/exim/DynaStop.Excludes.conf Include /usr/exim/DynaStop.Patterns.conf Include /usr/exim/DynaStop.NoMail.conf ### ### Lists from BleedingThreats.net and webhelper4u.net ### ### I make no warrenties about the stability of these lists. ### USE AT YOUR OWN RISK. Include /usr/exim/DynaStop.BleedingThreats.conf ### ### This is for the AutoGreen feature ### Include /usr/exim/DynaStop.AutoGreen.conf ### ### Tom Knaupp's list of False Positives ### Include /usr/exim/DynaStop.TomKnaupp.conf ### ### DNSWL WhiteList ### Include /usr/exim/DynaStop.WhiteList.conf ### ### Spamhaus DropList ### Include /usr/exim/DynaStop.DropList.conf
This file needs to be saved in the same directory as the DynaStop executable, i.e./usr/exim as DynaStop.conf.
Now we need to start DynaWall,
the DynaStop server:
( /usr/exim/DynaWall & )
Be sure to put the above line in your start up scripts before Exim.
Now that DynaStop is configured and ready, we need to configure Exim to use DynaStop. We need to know the return codes from DynaStop for Exim:
0 - Not dynamic/no error
1 - Dynamic address
2 - No reverse address
3 - An address listed as NO mail
4 - Greylisted IP address
5 - Some sort of error, most likely corrupt IP address
Signals that DynaStop (DynaWall Server) respond to:
| HUP | The HUP signal tells DynaWall to reload the configuration file |
| USR1 | The USR1 signal tells DynaWall to write the statistics and cache files to disk |
| QUIT | The QUIT signal tells DynaWall to shutdown and cleanly leave memory |
Now we can begin editing the Exim configuration file.
The following is suitable for testing and diagnostics. ALWAYS test your patterns.
Add ONE of the following to your acl_check_rcpt ACL:
To add a warning header to the message, add the following block:
warn set acl_m7 = ${run{/usr/exim/DynaStop localhost $sender_host_address}}
condition = ${if eq {$runrc}{1}{yes}{no}}
message = X-DynaStop: Dynamic IP address/$sender_host_addressTo block a message, add the following block:
deny set acl_m7 = ${run{/usr/exim/DynaStop localhost $sender_host_address}}
condition = ${if eq {$runrc}{1}{yes}{no}}
message = Dynamic IP Address: Please use your provider's designated mail exchange serverTo defer a message, add the following block:
localhost can be any hostname in your network or on the internet. If you use an external DynaWall server, be sure you check with that system's administrator first. Also, this must be a fully resolvable host name, not an ip address.defer set acl_m7 = ${run{/usr/exim/DynaStop localhost $sender_host_address}}
condition = ${if eq {$runrc}{1}{yes}{no}}
message = (D1) Please try later...
Be sure you test your patterns extensively before going to blocking mail. This is a powerful tool and can block all mail inadvertently.
NEVER put this above you abuse and postmaster entries in Exim's configuration.
To add IP lookup information for abuse:
warn condition = ${if !def:h_X-AbuseInfo:} message = X-AbuseInfo: http://dns.tanaya.net/?whois+$sender_host_address
Restart Exim and you're done. DynaStop is now integrated into Exim and filtering mail according to your settings.
At the top of your Exim configuration file, put the following lines:
DYNASTOP = ${readsocket{inet:localhost:337}{$sender_host_address\n}{60s}{}{-1}}
Here is an example of the Cardwell Macro in use:
### The Cardwell Macro
deny condition = ${if eq{DYNASTOP}{1}}
message = Dynamic IP address
For procmail, DSpam, or any other inline filtering method (messsage read from/to stdin/out), the IP address has to
be taken from the message. Note that GreyListing at this point serves no value and should NOT be used except at
the SMTP level. The command line is slightly different for inline filtering:
DynaStop localhost stdin
The stdin parameter tells DynaStop to get the message from standard input, extract the IP address, and
send the message to standard out with the X-DynaStop header set if appropriate and exit with a return code of
0. DynaWall must still be installed as this is the work horse of DynaStop. See above for installing DynaWall.
Procmail recipe for DynaStop:
Add these lines to the begining of your procmailrc (be sure to change paths as appropriate):
Add these lines to the filtering part of your procmailrc (change Spam as needed):
:0fw
| /usr/exim/DynaStop 127.0.0.1 stdin
:0
Procmail will now filter dynamic IP addresses as desired.
* ^X-DynaStop:.*D[1-3]
Spam