Started a WoW Blog

I started a blog just to jot down random stuff about my experiences playing WoW. It's really hard to work on data collection when it keeps you from playing the game at all. (I don't have a lot of spare time)

Big Red Pet

The Inevitable Hiatus Post.

If it wasn't obvious by now, I haven't spent a lot of time working on this lately. I'm firmly committed to discussing the finer points of builds (and point people once again to the largely empty forum I've set up for this type of discussion) so hopefully I can start up some discussion.

If you're particularly proud of your build, I highly encourage you to sign up on the forums and write up a post in your class forum about it. If we can get enough people to do this, discussion will surely follow.

New Build Shop up - Warlock ??/41+/??

WoW Insider has a new Build Shop up, this time focusing on a Demonology spec. This one's a bit open-ended, since the requirement is just 41+ points in demonology. I've updated the existing 70s I have now and fetched their (relatively) current character sheets. Scanning the top 20 Warlock specs on my current report, #5 is 7/43/11, and #6 is 20/41/0. Coming in at #16 is 18/43/0, #18 is 5/45/11, and #20 is 9/41/11.

Obviously, I've got a new report out, but I'm redesiging the report generation to be more general and flexible. The goal here is to support live queries against that database on various criteria. I know for a fact that making this live will kill my server, so I need to figure out a way to ensure that doesn't happen. In any case, you should eventually be able to do top N spec/builds per class, with these additional variables: faction, race, gender, realm, realm type and battlegroup. I think a lot of people will be interested in the realm type breakdown and if there truly is a difference between how PvE and PvP players choose their talents.

Standardizing Armory data storage

There are a number of projects out there to fetch and operate on Armory data. However, there doesn't seem to be much interaction between these projects. The simplest way to cache Armory data is to just fetch the 5 XML files for each character and store them as blobs in a table. However, that's not really conducive to doing complex queries and searching it doesn't scale. I'm working on a database representation of character data (arena teams is on the table next) and I'm wondering if anyone else out there is interested in working on this. The goal is to create a freely available database schema that any Armory-related project could take advantage of for reasonably quick offline searching. If you're interested, drop me a line at thebuildmine@gmail.com. Once the schema's done, then work can begin on a fetch and management library for various scripting languages. (perl, php, python, ruby, etc.)

Round 2 gathering has started

After going through the first grab of 2.5M characters, I've gone back and done some refinements on data gathering and optimization. I'm currently acquiring the latest data on all the characters I've gotten before, sorted by level descending, since what we're most interested in in the short term is data on level 70s. This time I'm fetching all of the armory data and storing it instead of just the character sheet for more complex queries and comparisons. Eventually I was get things to a point where live searches can be done.

Forums up!

I'm happy to announce that I've put up forums for discussion on the results discovered from datamining the Armory.

Racial Report

I respun the report and added the racial breakdowns. I don't have class numbers by race offhand, but I can run those this weekend. The original reports are there too, but they're the same as before since it's all scripts running against the same data. Over the next couple of weeks I'll be overhauling my schema and collection tools now that I've gone through this exercise once.

First Report

Ok, here's a link to the first "full" report. (I am fully aware that my web design fu is negative. Hopefully CSS will save me someday.) This structure of report should mostly stay the same (I hope). For each class I've generated 4 spec/build reports, one specialized for each talent tree (31 points or more in a given tree is considered specialized) and overall. Each report lists the top 10 specs, and the top 10 builds within each spec. There's way too much information for me to come close to discussing this, so if you find something you want to discuss, blog about it and let me know and I'll mention your blog here.

Build Results Layout - How do you want to see it?

So after a number of problems over the past month or so, I've got almost 2.5 million characters in the database. Live queries are out of the question, because my server just doesn't have the oomph for it. So for each class, I'm going to run the top 10 overall specs, and the top 10 builds of those specs, but I'm also going to run the top 10 specs (and top 10 builds within those specs) from each tree, to show a little more diversity specwise. This will take a while, but I'll try to have stuff up before the end of the month.

1st Million - Hunters

The number of level 70 hunters? 38,030, not the least, but far from the most, as detractors would have you believe.

1. 41/20/0 - 2221 (5.84%)
2. 0/43/18 - 1574 (4.15%)
3. 0/45/16 - 1450 (3.81%)
4. 0/42/19 - 1064 (2.80%)
5. 0/44/17 - 864 (2.27%)
6. 0/41/20 - 806 (2.12%)
7. 7/45/9 - 800 (2.10%)
8. 0/47/14 - 681 (1.79%)
9. 5/53/3 - 647 (1.70%)
10. 5/47/9 - 579 (1.52%)

The amount of variance in the Beast Mastery "cookie-cutter" spec alone is pretty staggering, although 23.6% of all Beast Mastery hunters have it, suggesting not much variance at the spec level. Humorously enough, BRK's current spec is not represented at all in the top 10 builds for this spec. I guess I haven't crawled Madoran yet, so it will be good to see how his build fares when all 3 million or so characters I have get pulled in. The top two builds differ only in the ubiquitous Tier 2 Marksmanship decision: Efficiency or Improved Hunter's Mark? Solo or Raid? 8 out of 10 builds put 2 points in the much maligned Spirit Bond (version 2). Another point of contention seems to be where to put the 5th point for Tier 7 in order to get to Serpent's Swiftness: Bestial Discipline is the most popular resting place, or else a token point Animal Handler or Catlike Reflexes.

As for the rest of the specs, they are completely dominated by Marksmanship builds. From the rather confusing 0/41/20 to the PvP-oriented 0/43/18 to what I consider to be even more degenerate than the "glass cannon", 5/53/3.

I don't even know where to begin. Less than 4% of the hunters that chose 0/41/20 are represented in its top 10, so I can only conclude those that chose this spec are either deluded, misinformed, or just plain ignorant. Diversity is a good thing, but when your top build is shared less than a percent of your spec-mates (as it is with two other specs in the top 10), it's overdoing it. Either that or I'm missing something that might present itself in a PvE vs. PvP server breakdown. Look for that in a future post.

5/53/3 is just plain offensive to me. (Warning, religious zealotry ahead) We as a class are not complete without our pet, so why go so far out of our way to gimp ourselves by gimping our pet? And please, repeat after me: Hawk Eye is a crutch. I'm far from the best hunter around, but I can function just fine without it. There are plenty of other talents that give you more bang for the point. (pun only partially intended)

Ironically enough, it is 5/47/9, the last of our top 10 specs, that has the highest percentage sharing a single build. 21.42% of the only 579 hunters that went this spec chose it, practically 3 times the most popular build of next in line, 7/45/9. (figures. :P)

Am I biased? Hell yeah. I am (well, will be, someday :P) part of the most popular spec for hunters, yet it only represents a small (yet extremely vocal) minority of hunters overall. (68.56% of level 70 hunters are Marksmanship, as opposed to 24.65% opting for Beast Mastery, and a whopping 6.52% are Survival. Even more frightening is the remaining 0.25% (90 of you, and 8 who cleared their talents and haven't respecced yet!) that are of no spec at all, with no possibility of a 31-point, much less 41-point talent.)

A lot of numbers here. In the end, you can't just look at the spec, or at the build level. Context is everything. Hopefully everyone's not too confused to comment.

What numbers do you want to see?

I've been posting about the kinds of numbers I want to run against the data, but the bigger question is, what sort of information do you want to find out about the particular builds of the US WoW population? Llew's run some numbers on the most popular talents, acrost builds specializing (spending 31 or more talent points) in a given tree. It might be worth looking at talent popularity regardless of specialization. Also, what about levelling builds? No one really seems to want to talk about those much, but running a comparison of builds at 20,30,40,50 and 60 could be enlightening. And just how many 60s are there still, anyway? (Roughly 45,000 out of the 900,000ish characters I have so far, roughly 5%. I know of at least 4 of them though that are waiting in the wings while I get my first character to 70... :P) Is it possible there are people who didn't buy the expansion? If you look at the shelves of my local Fry's, you would think no one at all stood in line hundreds deep to get The Burning Crusade. There are at least a hundred Collector's Editions alone, and untold more regular editions. But I digress. What is it you want to know?

Waiting for 1 Million

Sorry for the lack of posting, but I'm waiting for the system to get 1 million characters imported then I'll run a bunch of numbers for the various classes, with spec and build breakdowns, and some comparisons I haven't really seen run anywhere before, like build popularity on PvE/RP servers versus PvP/RPPvP servers.