Berlin Brown and Software Development

I am working for a web specification for the botlist frontend. The botlist backend and web frontend can operate independently. The backend sends data to the frontend and the web frontend is used to display that information. http://www.botspiritcompany.com/botlist/ In terms of specification. The primary goal is to display URL information from the botlist repository. For example. Specification 1: 1. Display links in a listed format 2. Have a link to the host name 3. Allow for user submissions That is the idea. Why have a specification? Well, because I believe in programming language agnostic oriented development. The web front end could be implemented with the Spring framework, Lisp, or Django. Actually, I plan on adding a Lisp/Django component in the future.

In the functional programming toolkit; foldl is pretty useful. In layman terms, it is basically a way to call a function on a given input list. From zvon.org; the haskell foldl is defined as: "It takes the second argument and the first item of the list and applies the function to them, then feeds the function with this result and the second argument and so on. See scanl for intermediate results." The type definition is: Foldl : (a -> b -> a) -> a -> [b] -> a Lets say that out loud; I can call the foldl function with the following parameters; the first parameter passed to foldl is a generic function that has two inputs a and b . This input function returns a type of a. The second parameter has a type of a , the first, initial item or the result of the previous operation. The third parameter is an array of type b. The listing below shows how you might use foldl. let a = foldl (\x y -> (++) x (show y)) "-->" [77, 88, 99] To make it more comp

Yes, that is my real signature; thanks inkscape.

What is the purpose of botlist? What is its role? http://botspiritcompany.com/botlist/ A lot of web applications provide you with information based on your input. Google and yahoo work in this manner. You pull information from them based on some query. Botlist hopes to operate based on a push architecture. Much in the way that other news aggregation services works. Information is piped through to the botlist web application and the top articles and links are displayed to the user. The other links that aren't as popular will still be available through search. Pretty simple, yet a lot of work. The back end crawler is built with a combination of haskell and python. Haskell for the content analysis and python for the crawler.

It is now out there. http://paulgraham.com/arc0.html "We're releasing a version of Arc today, along with a site about it at arclanguage.org. This site will seem very familiar to users of Hacker News. It's mostly the same code, with a few colors and messages changed."

This listing shows how to open a file, extract the contents, split by the delimiter (regex is a little off) and then mashall into a datatype. data PageURLFieldInfo = PageURLFieldInfo { linkUrlField :: String, aUrlField :: Integer, blockquoteUrlField :: Integer, divUrlField :: Integer, h1UrlField :: Integer, imgUrlField :: Integer, pUrlField :: Integer, strongUrlField :: Integer, tableUrlField :: Integer } -- -- The info content file contains html document information. -- It may not exist but should, also contains URL info. readInfoContentFile :: String -> IO PageURLFieldInfo readInfoContentFile extr_file = do let extr_n = (length ".extract") extr_path = take ((length extr_file) - extr_n) extr_file info_file = extr_path ++ ".info" -- Extract the file, in CSV format. -- URL::|a::|b::|blockquote::|div::|h1::|h2::|i::|img::|p::|span::|strong::|table csvtry <- try $ readFile info_file -- Ha

Given an input string, output with a string that that has been sanitized. Example Input: A AAND ABLE ABOUT ABOVE ACQUIRE ACROSS AFOREMENTIONED AFTER AGAIN AGAINST AGO AGREE AGREEABLE AHEAD AKIN ALBEIT ALCUNE ALGHOUGH ALGUNOS ALL Example output: aand able about above accordance according acquire across aforementioned after again import qualified Data.Set as Set import Data.List wordTokens :: String -> [String] wordTokens content = tokens where maxwordlen = 100 lowercase str = map toLower str alltokens = splitRegex (mkRegex "\\s*[ \t\n]+\\s*") (lowercase content) tokens = filter (\x -> length x > 1 && length x < maxwordlen) alltokens -- -- -- Given an unclean content set; tolower, filter by length, get unique tokens, -- tokenize, join the list

It doesnt matter about the drugs and rock and roll, sex That was an example test case that I used against my bayes classification application. Bayes and other probability algorithms are used in most of the spam filtering software that is out in the net. I use various probability routines to classify the news articles. I begin with Toby's python code on the subject and ported is examples to haskell. So far, the results have been pretty satisfactory. The naives bayes classification is dismal, but maybe I am looking at the numbers wrong. In classic form, I will throw code at you and will give a lollipop to the mind reader that can figure out what is going on. The output of the test case: In my first test case, I extracted a couple of articles from the web with these subjects; politics , business , entertainment . A human can easily tell what is going on. A software program is going to have a lot of difficultly for several reasons, but two main ones. One, there are too many ST

http://www.autonomiccomputing.org/ "What is autonomic computing? It is the ability of systems to be more self-managing. The term autonomic comes from the autonomic nervous system, which controls many organs and muscles in the human body. Usually, we are unaware of its workings because it functions in an involuntary, reflexive manner -- for example, we don't notice when our heart beats faster or our blood vessels change size in response to temperature, posture, food intake, stressful experiences and other changes to which we're exposed. And, by the way, our autonomic nervous system is always working." Alan Ganek, VP Autonomic Computing, IBM

It is always useful to manipulate files. This set of code (runTestTrainBayes) gets all the files in the relative directory path "train"; calls the anonymous function on each file and reads the content; appends the content into one big string. workTokens returns only tokens that are greater than 1 and less than 100. Eventually, you end up with a list of word tokens. import System.Directory (getDirectoryContents) import List (isPrefixOf, isSuffixOf) import Data.SpiderNet.Bayes trainDir = "train" wordTokens :: String -> [String] wordTokens content = tokens where maxwordlen = 100 lowercase str = map toLower str alltokens = splitRegex (mkRegex "\\s*[ \t\n]+\\s*") (lowercase content) tokens = filter (\x -> length x > 1 && length x < maxwordlen) alltokens runTestTrainBayes :: IO () runTestTrainBayes = do putStrLn "Test Train Bayes" files <- getDirectoryContents trainDir let trainfiles = f

"This year I only read 70 books, down from over 120 last year. I guess that's not too bad considering this was the Year of Other People -- I still beat my general goal of 52. Once again, here are the books that struck me as completely worth reading. This year, though, I've intermixed them with the other books I read:" http://www.aaronsw.com/weblog/books2007 Aaron Swartz is well read; he said he read 70 books and 120 the previous year. I can't top that. But I do read a lot of technical books and like him only interested in government in politics. So there are only two categories. And normally technical is typically software/programming related. Here is my list; I didn't completely read all of them, "DNA String algorithms" doesn't exactly make for sit down type of reading material. Technical Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology Programming Erlang: Software for a Concurrent World by Joe Armstron

The project will start using XUL for database tools. http://developer.mozilla.org/en/docs/XULRunner

This is major news. The Lucene developers released Lucene 2.3; in future releases except machine learning capabilities. http://www.infoq.com/news/2008/01/lucene-23-mahout "That will be a separate project, but may be beneficial to Lucene users. There are currently some patches in JIRA for Lucene that implement ML algorithms. The goal of this project is to provide commercial quality, large scale machine learning (ML) algorithms built on Hadoop under an Apache license. I have seen a fair amount of interest already, and hope to have this project underway in the coming month." http://ml-site.grantingersoll.com/index.php?title=Incubator_proposal Would be interesting if the team could introduce the following algorithms. * 1.3.1 Naive Bayes * 1.3.2 Neural Networks * 1.3.3 Support Vector Machines * 1.3.4 Logistic Regression * 1.3.5 Locally Weighted Linear Regression * 1.3.6 k-Means * 1.3.7 Principal Components Analysis * 1.3.8 Independent Component Ana

I can use the Haskell programming language to better expressive the nature of the software task that I am attempting to accomplish. Let me reword that so the previous sentence makes sense: I use Haskell to accomplish complicated programming tasks; the expressive nature of the language provides an effective conduit to make that happen. One more time: Haskell is readable, fast, and expressive. I like it.

Pizza meals are effective. The pizza delivery person comes to your home. You pay the person. You eat the food. A pizza is circular in shape. Some of my favorite toppings include jalepenos, mushrooms, onions, beef or sausage; no more than two toppings at a time. Tacos are also very appetizing.

http://www.realworldhaskell.org/blog/ I am a rare breed and love picking up technical books. Some say they are too behind the technology trends. That is true, but if you need a book on text parsing or artificial intelligence, then a 30 year old book is as good as a 2 year old one. The real world haskell book is one that I am looking forward to; I have already picked up two haskell books and this will only enhance the collection.

Here is a simple code snippet that shows the variation between how to approach problems in python and how to approach problems in haskell. More often than not, the same problem or task that you want to accomplish is the same but the methods used are going to be different. There is no question that python is an imperative language. You will tackle a problem by figuring how how to perform step 1 and then step 2, possibly N number of times. Haskell is a purely functional programming language but some like Simon Peyton Jones call Haskell "the best imperative language". Perform the invchi2 calculation in python example_data = [ (4.3, 6), (2.2, 60), (60, 2.2), (0.3, 4), (32.123, 20), (12.4, 5), (0.04, 3), (0, 0), (1, 1) ] def invchi(chi, df): m = chi / 2.0 sum = term = math.exp(-m) for i in range(1, df//2): term *= m / i sum += term return min(sum, 1.0) if __name__ == '__main__': print "Runnin

I am adding support for text analysis including bayes classification. The code is based on Toby Segaran's PCIL python source code and contains some of those utility functions. Libraries used: import System.Environment import qualified Data.Map as Map import qualified Data.Set as Set import Data.List import Text.Regex (splitRegex, mkRegex) Type definitions: type WordCat = (String, String) type WordCatInfo = (WordCat, Int) type WordInfo = (String, Int) Utilities for finding the word frequency in a document: -- -- | Find word frequency given an input list using "Data.Map" utilities. -- With (Map.empty :: Map.Map String Int), set k = String and a = Int -- Map.empty :: Map k a -- foldl' is a strict version of foldl = foldl': (a -> b -> a) -> a -> [b] -> a -- Also see: updmap nm key = Map.insertWith (+) key 1 nm -- (Original code from John Goerzen's wordFreq) wordFreq :: [String] -> [WordInfo] wordFreq inlst = Map.toList $ foldl' updateMap

Once again, a post not my own. This is from the comp.lang.lisp forum; a discussion on Rails. And I tend to agree: A quote from MacieJ. "First, I completely agree with what Slava said. Now to expand that a bit with my own thoughts: Rails is 100% magic with 0% design. It sports all the great quality and consistency you've come to expect from PHP, except with loads more magic. There's no overarching design or scheme of things, it's just a bucket of tools with some glue poured in. This has a couple of important consequences: - There's no reasoning about Rails -- more familiarity won't give you better chances of figuring out something new because that's what follows from the design, only because that's how it usually ends up being implemented and because you have memorised more things, so your guesses are better. In essence, being better in Rails means being better at grepping the internet. - There's no thought given to the general problem to solve, it&

In my previous blog post, I tried to describe some mainstream programming languages in a couple of words. After some deep reflection, I am revising what I said earlier. This time, I want to associate the language with tasks that the language is known for. 1. Haskell: General Purpose Language 2. Python: Twisted network framework, scripting, easy to read, easy to use 3. Ruby: Ruby on Rails, dynamic, easy to use 4. Scala: Improvement over the java language, work with existing java libraries 5. Java: Lots of libraries, lots of community, commercial involvement 5. Factor: New and emerging, fast, smart people working on the project. 6. C: Desktop development, fast, low-level development. 7. Erlang: Parallel development, server development

Just thought I would share. Haskell is just plain awesome.

As part of the botlist text content mining backend I am working on; I frequently process word tokens in a web page document. The code below contains utility functions for creating a data structure that consists of a list of tuples. The tuple contains a word and the number of occurrences in the document. I modified John Goerzen's original wordFreq version from http://changelog.complete.org/plugin/tag/haskell. import System.Environment import qualified Data.Map as Map import Data.List import Text.Regex (splitRegex, mkRegex) -- -- | Find word frequency given an input list using "Data.Map" utilities. -- With (Map.empty :: Map.Map String Int), set k = String and a = Int -- Map.empty :: Map k a -- foldl' is a strict version of foldl = foldl': (a -> b -> a) -> a -> [b] -> a -- (Original code from John Goerzen's wordFreq) wordFreq :: [String] -> [(String, Int)] wordFreq inlst = Map.toList $ foldl' updateMap (Map.empty :: Map.Map String Int)

[1.] Function Calls in Haskell As you may have noticed, passing input to a function in haskell is simple because of the statically type nature of the language. A function can have a complicated input and output return, but at least you are given the parameters of that particular function. In python you can pass anything to a function; you can pass a function, a integer constant, an instance of a class and yet the function really only expects an boolean. [2.] Examples putStrLn :: String -> IO () For example, putStrLn expects a string as an input and will return unit in the IO Monad. >putStrLn "abc" Will print abc to the console. Prelude> putStrLn "abc" ++ "123" :1:0: Couldn't match expected type `[a]' against inferred type `IO ()' In the first argument of `(++)', namely `putStrLn "abc"' In the expression: putStrLn "abc" ++ "123" In the definition of `it': it = putStrLn "

Here is a a one or two word summary of the languages I work with. I try to spend time with most of them. They are not listed in any particular order. 1. Haskell: Functional, cool [most projects] 2. Python: Dynamic, semi-fast, indented blocks [most projects] 3. Ruby: Dynamic, semi-slow [some projects] 4. Scala: Functional, better java, actors [some projects] 5. Java: Static, lots of libraries, ugly, verbose [most projects] 5. Factor: Cool, new, fast [a couple of projects] 6. C: Cool, sloppy [a couple of projects] 7. Erlang: Parallel [very few projects] (Less used): 7. C++: scary (I don't use C++) 8. Common Lisp (sbcl): fast, simple, dynamic 9. Scheme: simple, dynamic (Useful): 10. Bash: ok I guess (Want to learn): 11. Perl6 12. Relearn Lisp

What defines a web page; if you had an input/output device. What input would define a page and what would the output be? I am currently searching for different tools and algorithms to analyze pages that are available on the internet. Presented here in this chart are various html tags (e.g. table, strong, bold) and the number of times the tag appeared in the document. 5 page documents were analyzed; 3 good article pages from wikipedia and 2 spam pages. The link below contains the data that generated the chart above. http://openbotlist.googlecode.com/svn/trunk/openbotlist/docs/media/content/chart_content.txt

I always forget this simple idiom for removing a string based on a set of words to remove. This particular example, given an array of stop words, remove them. BOTLIST_STOP_WORDS = [ "abc", "123" ] keywords = "kjskldjfkljskdfksdkl abc lkjsdfksdklfd 123123" def filterStopwords(self, keywords): """ Find all of the stopwords and remove them""" res = [] keywords_list = keywords.lower().split() res = list(set(keywords_list).difference(set(BOTLIST_STOP_WORDS))) # Return the new keyword string return " ".join(res) In that example, the stop words will be removed from the string.

Bach, Brandenburg concertos; 4, 5, 6: This particular rendition features a pipe, string ensemble that is a mix of slow and spirited; but never dry or boring. This particular set seems to be more oriented towards strings than 1, 2, 3. I could be wrong about that though. So far, Bach has impressed me the most. http://en.wikipedia.org/wiki/Brandenburg_concertos

I am joining and now following the community at theinfo.org. It is exactly what I have been looking for in terms for large data set crawling. http://theinfo.org/

This person seems pretty fascinating, Hunter Thompson. My only knowledge of him is that he created Fear and Loathing. http://en.wikipedia.org/wiki/Hunter_S._Thompson Hunter Stockton Thompson (July 18, 1937 – February 20, 2005) was an American journalist and author, famous for his novel Fear and Loathing in Las Vegas. He is credited as the creator of Gonzo journalism, a style of reporting where reporters involve themselves in the action to such a degree that they become the central figures of their stories.

Announce: This database will be used as the simple binary file format for queuing requests. Ideally, rabbitmq would be be used but the client has not been implemented. (Basically, this is just a simple file format; if you want to see the haskell source, see the link below. Covers Data.Binary use, ByteStrings, decodeFile, encodeFile and some other basic binary file manipulations) instance Binary SpiderQueue where put dbq = do BinaryPut.putWord16be (magicNumberA dbq) BinaryPut.putWord16be (magicNumberB dbq) BinaryPut.putWord16be (majorVers dbq) BinaryPut.putWord16be (minorVers dbq) BinaryPut.putWord32be (queueSize dbq) -- @see mapM: Monad m => (a -> m b) -> [a] -> m [b] (mapM_ put (queue dbq)) get = do magicnumbera <- BinaryGet.getWord16be magicnumberb <- BinaryGet.getWord16be major <- BinaryGet.getWord16be minor <- BinaryGet.getWord16be len <- BinaryGet.getWord32be -- ****

A quote from: Tim Sweeney - Re: Hundred Year Language http://lambda-the-ultimate.org/classic/message6475.html#6501 All the content in this post is a quote about various languages out there. "C# is a great illustration of the superficiality of popular language progress. It is the most polished and fine-tuned mainstream language ever created, yet fails to come to grips with basic mathematics type and theory. You still have absurdities like 2^64=0, 7/2=3, simple expressions like x/y and a[x] which the compiler accepts yet aren't actually typesafe in any reasonable definition of the term, arrays have serious typing flaws, and the basic confusion between things and references to things remains. Right now, the open language lineages are: - LISP: This is the apex of untyped programming with nested scoping and metadata manipulation. It's at the end of the line, not because of lack of potential, but it's so simple and powerful than any improvements to it can be implemented in L

For those us that haven't done calculus or differential equations in a while, visualizing functional composition in functional languages like haskell may be difficult. So here are some practical examples for really visualizing composition. The wikipedia definition for function composition in a computer science context has the following: "function composition is an act or mechanism to combine simple functions to build more complicated ones." In math 101, we learned the following: given the value of 'x' , pass 'x' to the function g and then the result of g of x to the function f to get a result. f(g(x)) Let's say we are doing the following calculation: y = g(x) = 2 * x z = f(y) = 3 * y z is our result. If we provide an input of 4, we have the following. y = 2 * 4 = 8 z = 3 * 2 * 4 = 24 z = 3 * g(x where x = 4) = 24 In Haskell How would you do this in haskell (in prelude): Open ghci: First, here is the function definition for function composition: (.)

Well, I have made a little bit more progress with the simple haskell network client and connected to a RabbitMQ server and received some of a valid response. What makes this example a little bit different than a simple echo client/server connection is that a AMQP client needs to send valid byte responses in big endian format. For example, the first 1 octet that must be sent to the server include this header BYTE:'A', BYTE:'M', BYTE:'Q', BYTE:'P', BYTE:1, BYTE:9, BYTE:1, BYTE:1. Bytes are easy, but sending short values and also sending unicode responses requires a little bit of work. The java client library provided by RabbitMQ as you might have guess makes sending unicode or byte packages straightforward. In haskell, well there just isn't a lot of code out there that demonstrates how to do this. With that being said, here is some update to the simple client. At first I wanted to just demonstrate networking in haskell, but now I may go a step f

I am building a AMQP client library to use along with some of the openbotlistprojects. The client library will need to be written in haskell and will be able to communicate with MQ servers like RabbitMQ. That is irrelevant because I am just wrapping my head around haskell networking. The "Data.Binary" module is used here to have fully control over the size and endianess of all data sent across the network. There are several utilities for writing shorts, byte words, longs, 64 bit longs, etc. Here is a data structure defined with the Binary.Put monad. Our task is write this data to the socket connection. data AMQPData = AMQPData { amqpHeaderA :: [Word8], amqpHeaderB :: Word32 } instance Binary AMQPData where put amq = do BinaryPut.putByteString (pack (amqpHeaderA amq)) BinaryPut.putWord32be (amqpHeaderB amq) amqInstance :: IO AMQPData amqInstance = return (AMQPData { amqpHeaderA = (unpack (C.pack amqpHeadA)), amq

First, thanks to 808140, he did a good job of explaining monads. This is not my content at all, but his. A quote from reddit, 808140: "The first step to understanding "monads" is realizing that the name is scary but the concept is simple. See, the term itself comes from math and was so-named because the people that found a use for it in programming were math people. Despite what some people seem to think, though, you don't need to be a math person to understand them. The best way to grok monads is to not try to understand what they are abstractly first. Instead, familiarize yourself with several common monads and their uses. This will allow you to better see what is being abstracted. A lot of people who are new to monads try to understand the abstraction before understanding the specific cases. Some people (usually people with math backgrounds) can do this, but for most people, the best way to understand something abstract and general is to start with specific e

I have made some small updates to botnode.com, looks a little prettier than what was existing. The purpose of the site is up for debate, but will normally center around all things software development. http://www.botnode.com/

Specification for Emacs Link Blog Tool. Overview: A link blog contains blog entries which in turn contain a list of links. This tool will aid in the creation of a link blog service. The emacs IDE is a powerful development environment that also includes a lisp based plugin system. We can build a Text over HTTP protocol that interfaces with an emacs plugin. A user can then use the emacs plugin to add entries to the blog.

http://www.lsi.upc.edu/~nlp/SVMTool/

Support Vector Machines and Document Classification Saurav Sahay "Automatic Text categorization using machine learning methods like Support Vector Machines (SVM) have tremendous potential for effectively organizing electronic resources. Human categorization is very costly and time-consuming, thus limiting its application for large or rapidly changing collections. SVM is a comparatively new technique with a very solid mathematical foundation for solving a variety of ‘learning from examples’ problem and gives high performance in practical applications." Resources http://www-static.cc.gatech.edu/~s sahay / saurav sahay 7001-2.pdf Keywords Reuters 21578, SVM

Kind of interesting to write lisp like code in haskell. Note, all of this is valid haskell: Prelude> ((+) ((*) 1 2) 10) 12 Prelude> (1 * 2) + 10 12 Prelude Text.Printf> (printf "%d\n" 5) 5 Prelude Text.Printf> printf "%d %d\n" 1 2 1 2 Prelude Text.Printf> ((printf "%d %d\n" 1) 2) 1 2 Of course, you have to match up the arguments in haskell accordingly; not all functions take a just a list like you might find in common lisp.

There post doesn't contain much content, but I thought that this Haskell List reference sheet gives a good overview of some of the powerful list functions in haskell. http://www.haskell.org/ghc/docs/latest/html/libraries/base/Data-List.html

The syntax left arrow denoted by "<-", allows you to bind a variable from the result of an IO computation. Saizan on freenode does a better job of explaining it... When asked; What are the main differences between left-arrow and binding a variable with let. "the <- arrow "extracts" the value from an IO computation, while using let you'd just define another name for the computation as a whole" "so in let x = getChar, x :: IO Char, but in do x <- getChar; ... x :: Char"

I have been defending Ron Paul on his blurb about evolution. Here is one response in the reddit forum: I am still waiting on where he said he (Ron Paul) doesn't believe in "EVOLUTION". Big difference from not siding with EVOLUTIONARY "THEORY" in which you aren't 100% on the entire details on how human beings arrived here today. I am a life long person of science. Lets understand evolution as fact and push everything else away. But I can understand how a Christian can take issue with an some established origins of life; how some out of the 6 billion people can be confused about the origins of life. If you can give me the video where Ron Paul says the following "I DO NOT BELIEVE IN EVOLUTION AT ALLL, WILL NEVER BELIEVE EVOLUTION" or "I AM A CREATIONIST FROM THE DAY I WAS BORN TILL NOW" Yes, I saw the youtube video mentioned earlier, but you can play all kinds of "Bill Clinton wordisms" with that particular Ron Paul blurb. Al

"Abdel Benamrouche announced that he has updated the original 0.01 Linux kernel to compile with GCC-4.x, allowing it to run on emulators such as QEMU and Bochs. After applying his series of small patches, Abdel explains that the 0.01 kernel can be built on a system running the 2.6 Linux kernel. He added that he's successfully ported bach-3.2, portions of coreutils-6.9, dietlibc-0.31 (instead of glibc), bin86-0.16.17, make-3.81, ncurses-2.0.7, and vim-7.1 all to run on his modified 0.01 kernel." -- Quote from blog entry. http://kerneltrap.org/Linux/Dusting_Off_the_0.01_Kernel

It looks like CouchDB has taken off, in fact Damien Katz will now be working with IBM full time on CouchDB. This is a big deal to a developer, especially an opensource one. I don't know the true origins of CouchDB, but just looking back at his blog, it look likes like it got started in 2006/6. http://damienkatz.net/2006/06/ Congratulations.

There is a hackage haskell package that is external from the core haskell libraries that includes support for reading writing binary files and support big/little endian encoding and decoding. I used this, in order to manipulate the botlist binary database format. It wasn't intuitive as I didn't quite get the understand the Binary.GET monad which contained the getWord16le, getWord16be functions and how it related to the pure Binary monad, but after a while it made it sense. http://hackage.haskell.org/packages/archive/binary/0.4.1/doc/html/Data-Binary.html Download and review this haskell source to see what I ended up with. I defined a DatabaseFile type which includes two magic number words (unsigned shorts of 16 bits each) and then populated the fields from the binary file. http://openbotlist.googlecode.com/svn/trunk/botlistprojects/botspider/spider/tools/misc/dbreader/haskell instance Binary SpiderDatabase where put _ = do BinaryPut.putWord16le 0 get = do magi

Best and Worst of 2007, from a developer's perspective: Best of the Best : Reddit, Factor, Haskell Best Website: Reddit.com Best Movie: 3:10 to Yuma Best Food: Mexican Food (love the tacos!) Best new project, you probably haven't heard about: Factor Another best language, that is gaining a lot of traction: Haskell (honorable mention for Scala) Best Project: Lucene/Nutch/Hadoop Best Printer: Brother HL2040, works with linux (at least so far) Best Politician: Ron Paul Worst Product: Firefox2-3 on Linux. (Actually, flash plugins have a lot to do with the trouble on linux, but still). (Honorable mention for HP printers) Worst developer discussion forum: Joel on Software, http://discuss.joelonsoftware.com/?joel when you get a bunch of VB developers together, this is what happens. Worst websites: Twitter.com, Facebook.com, MySpace.com Worst political websites: Slate.com, Huffingtonpost, DailyKos,

Here is some python code for loading the java binary class format. It only returns the first couple of bytes and will only convert the bytecode format into a data structure. It won't parse the data. Maybe we can look at that in the future. Some useful python libraries: http://docs.python.org/lib/module-array.html "struct -- Interpret strings as packed binary data" http://docs.python.org/lib/module-struct.html Source: http://jvmnotebook.googlecode.com/svn/trunk/misc/bytecode/ http://java.sun.com/docs/books/jvms/second_edition/html/ClassFile.doc... Another note, the java bytecode class is big-endian format and most of us work on little endian machines these days, just something to keep in mind.