logo
home
datafeed
about
usage
mirrors
rss feeds
help us
links
contact
faq

 

Datafeed FAQ



How do I apply for the Datafeed service?

In order to request a data feed, you must first register an account. Once you have your account created and confirmed, you need to log in and then request a data feed. You are not committing yourself to anything by requesting a data feed. We will contact you by phone or email to verify the information you have supplied, and confirm pricing. Datafeed service for Year 1 is for 12 months. You may cancel your request at anytime within the first 30 days. If we have not received a cancellation notice and have not received payment, your data feed will expire automatically and be disabled.


Why is there a charge for this service?

Due to the administrative burden of relying on hardware donations from 3rd parties to provide infastructure support, URIBL moved away from this model and now maintains its own hardware/infastructure resources. Because of the costs involved in doing so, there is no way this service would remain "free-for-most" if we did not require the heaviest users to pay.


What if I already pay for Anti-Spam Hardware/Software that uses URIBL?

If you mail volume is low, we really don't care if you query the public mirrors. But if your hardware or software is hitting our public mirrors with high volume, then we will block your queries. At that point you can either continue without URIBL service, or request Datafeed Service. Feel free to raise your concerns with your vendor, as we would be happy to work with them to provide their own resolvers for their customers to hit.

The same applies for free software. If you are using SpamAssassin, then great. Since URIBL is part of default SpamAssassin installs, you automatically benefit from our service. However, if you run a large mail system with SpamAssassin, then there is a chance we will block your queries on the public mirrors. We understand you may not realize you are querying URIBL since it is enabled by default, and we will take the necessary steps to notify you, if possible, before blocking your queries from our public mirrors. If your queries are blocked on the public mirrors, and you use SpamAssassin, you will see a URIBL_BLOCKED rule firing in your email headers. For more detail on that, please read DNS Query Refused


What do I need to get started?

Datafeed over DNS - If you have subscribed to Datafeed over DNS, all you need is to either whitelisted your nameserver IP addresses through our Datafeed ACL interface, or use the custom DNS hostname we have provided you. In the case of the latter, you would need to change your Anti-spam software from looking up queries against multi.uribl.com, and replace that with your own custom hostname. This custom hostname will bypass any high volume blocks we have in place, allowing you to send your queries from the DNS server of your choice. So if you use public DNS resolvers such as OpenDNS or Google DNS, using the custom hostname is the only option.


Datafeed over RSYNC - In order to synchronize the data feed, you will need rsync. Most Unix and Linux distributions will contain this software automatically. If it is not installed, and your distro supports a package manager, try apt-get install rsync (DEB) or yum install rsync (RPM).

Once you have rsync installed, you will need to install rbldnsd to load up the zone files. Just as with rsync, you may try to install it using the package manager for your distribution.

If you are attempting to run this under a windows environment, both rsync and rbldnsd have windows versions. See wrbldnsd.

We will supply you with a synchronization script for the data feed. Please contact datafeed@uribl.com if you need technical assistance.


How should I configure DNS forwarding?

Our data feed service ships zone files without the SOA and NS headers normally found on rbldnsd zones. We do this to force configuration under your own zone, as misconfiguration normally results in the queries hitting our public mirrors.

For rbldnsd configuration, you will need to use a combined dataset. Your main rbldnsd file will contain the necessary Start of Authority (SOA) and Nameserver (NS) headers to point the resolution to your local DNS nameservers. For example, your rbldnsd startup would contain a line like this..

 /usr/sbin/rbldnsd -f -n -u rbldns -b 1.2.3.4/53  \
  -r /var/lib/rbldns -w / -p /var/run/rbldns/rbldns.pid \
  -l +log/query.log -s +log/log.stats \
  uribl.local:combined:zones/uribl.local.rbldnsd,zones/multi.txt,zones/black.txt,etc...

In this setup, your uribl.local.rbldnsd file would contain the $SOA and $NS defintions, and the reset of the datasets would be predefined. Your queries for multi would then look like domain.tld.multi.uribl.local, and your nameservers would need to forward queries for multi.uribl.local to the IP address where you have configured rbldnsd. On bind, this would look like..
zone "multi.uribl.local" {
        type forward;
        forward only;
        forwarders {
                1.2.3.4;
        };
};
Then, you would need to make sure your anti-spam software is pointing lookups to multi.uribl.local vs multi.uribl.com. NOTE: If you want to use "uribl.com" forwards instead of "uribl.local", please make sure you use the forward only; option in bind. If you're queries leak to our public mirrors due to misconfiguration, we will refuse the queries.



Can I run rbldnsd and bind on the same server?

Yes, and if you are running bind 9.x, you can even run them on the same IP address. If you are running bind 8.x, then you will need to bind a secondary IP address to the server and let rbldnsd listen on the new IP, as bind 8.x does not support forwarding to non-standard ports.

For Bind 8.x

Lets say 1.2.3.4 is your server IP on eth0. Bind 1.2.3.5 to eth0:0, and have rbldnsd use this new IP by specifying that IP in the -b startup option.

rbldnsd -b 1.2.3.5 ...

Now, edit your named.conf and forward the rbl queries to the new ipalias you have created on eth0:0.

zone "multi.uribl.local" {
        type forward;
        forward only;
        forwarders {
                1.2.3.5;
        };
};

For Bind 9.x

Start rbldnsd on a non-standard port by using the -b [ip]/[port] option.

rbldnsd -b 1.2.3.4/5353 ...

Then, add a forwarding zone in your named.conf to forward queries to your rbldnsd server on port 5353.

zone "multi.uribl.local" {
        type forward;
        forward only;
        forwarders {
                1.2.3.4 port 5353;
        };
};


Do I need the Datafeed Service?

If either of these conditions are met:

1) you provide commercial services which query URIBL
2) you generate over 100k queries a day

you need to subscribe to commercial datafeed service, either Datafeed over RSYNC or Datafeed over DNS. In order to generate over 100k queries, a mail system would need to see close to 100k emails per day due to average URI lookups per email. We will measure your average DNS query volume during a free trail period to determine actual usage. If you are trying to reduce your query volume to remain in the free tier, supressing unccessary queries via a uri_skip_list is a good method, as is operating a local caching nameserver and overrriding our DNS time-to-live. Note that overriding the TTL can reduce your spam accuracy for new campaigns that have negative cache at the start.


We cannot setup/maintain our own rbldnsd solution, are there other options?

If you have been blocked from sending queries to our public mirrors and you cannot setup a local mirror, we now offer Datafeed over DNS. See Requesting the Datafeed Service and choose Datafeed over DNS on the request form.


How do I know how many queries I'm sending to the public mirrors?

Normally you can base the number of queries on how much mail you do. If you are receiving 100k messages a day, 50-75k queries normally will hit the public mirrors. A caching nameserver may reduce that slightly, but with the low TTL times on the public nameservers, one will normally only see less than 10% of the queries come from dns cache.

If you are worried about being ACL'd from the public mirrors, then you are probably a good candidate for Datafeed Service. However, we will make an attempt to contact the IP owner before we block any queries. If you suspect you have been blocked from public DNS, see Blocked Query Testing for information on how to test to see if your DNS resolution has been blocked.


How long does it take to remove the block after I've subscribed to Datafeed over DNS?

Once your Datafeed over DNS account has been activated, and you have added your nameserver IPs or subnets to the Access List, you queries will begin to work in as little as 15 minutes! If the block recently took effect, you might have to wait up to 24 hours for the TTL to expire on your local DNS cache. For those with access to flush their own DNS cache, this would remove the 24 hour wait.


What hostname should I use to query Datafeed over DNS?

Once your IP addresses are added to your Access list, those IPs are propogated out to all our RBL zones. So, you can continue to query "multi.uribl.com" as configured by most anti-spam softwares (ie SpamAssassin), or you can update that to ".df.uribl.com", which is basically the multi zone with the additions of Gold (127.0.0.16), black_a (127.0.0.32), black_ns (127.0.0.64), and black_nsip (127.0.0.128). Note, additional rules or configuration may be necessary to take advantage of these additional return bits in your anti-spam software. See this link for usage examples, or contact datafeed@uribl.com for assistance.



Why are DNS queries from my cloud instances (AmazonEC2/Softlayer/Rackspace/etc) blocked?

Large subnets owned by Amazon and other cloud providers have been blocked due to high volume. Because amazon has so many networks, a single user may have multiple mail exchanges on multiple networks, and we have no ability to correlate this and block individual high volume users. We are looking at ways of improving our query limit system for those coming from large virtual hosting providers such as Amazon, but at this time we do not have anything in place. We do offer discounted Datafeed over DNS rates for low-volume, cloud hosted users who are effected by these wide ranging blocks. See Requesting the Datafeed Service and choose 'Cloud Hosted' on the request form.



Can you allow/whitelist my low query volume IP address

We cannot make exceptions for individual IP addresses to override a block of a larger subnet. You should check with your provider first, as they may provide alternate DNS servers to you. As a last resort, you would need to subscribe to Datafeed Service as a work around.