Friday, August 3, 2007

2nd Fetch update - 416k and counting

After 3 days or so, I've fetched new data on 416,000 of the 2.5 million characters I have so I should have new data by the end of august. I need to figure out the best way to multithread this process so I can grab data faster. As a side note, the XML data for those 416,000 characters is taking up close to 4 gigs of drive space. Coming up with a more efficient storage method is next on the list.

No comments: