[TOR] How to Handle Beta Feedback

I found this over on Darth Hater and thought it was an interesting way to handle beta feedback.

The use of heat maps, chat logs (to catch issues people talk about but don’t bother to /bug) and other various feedback items tagged for more immediate consumption makes good sense. Use of behavior analysis is neat along with the fog of war as an indicator for areas players generally miss.

I’m really stunned by the analytic techniques they’re using. I guess something good can come from all the targeted marketing algorithms and practices.

On the other hand… big brother is watching.

Q: How do you stop or catch leaks?

A: With all the data tracking we have we can see a single ability used at a single location for every player. If a leaked video shows a player wearing a certain piece of armor or doing donuts on a speeder in a certain zone we can find them very easily.


EDIT: Link to Gamasutra coverage of the same information.

This sounds a lot better and as a tester, I have to go clean my pants now.

How does user feedback influence content creation for The Old Republic? How do you gather this data?

Testing and the use of data generated from testing has been an integral part of our workflow for more than a year now and has been critical for us in validating the game design, rooting out problems and improving the overall game.

Data is gathered via a broad set of methods, including automation, very high detail metrics about user interaction with the game, professional focus testing, in-game player feedback systems, private testing forums and direct contact with individual testers or entire groups via chat.

It’s possible for us to drill down into the game interactions of every single tester and correlate their feedback directly with issues encountered in-game. By using a several different data sources, we can eliminate a lot of the usual bias encountered in direct user feedback.

High detail user interaction metrics also help us analyze complex content issues, develop fixes and most importantly, validate the success of those fixes a few builds down the road. 

I’ve been begging my developers (I has developers) to put something like this in for awhile now. When given the ‘not enough time’ excuse I suggested it was already there (mostly) in the Undo stack – just allow us to see it or get a dump some how. This would greatly increase the value of less experienced testers supplying feedback (like interns or contractors new to the product).

They still didn’t do it… bastids.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s