(An attempt at) The top 40 TPA players from leaderboard scores

[invitro]

Ok, I see this thread....
and I am equal parts amazed and horrified, and I think it may be the most amazing thread I've yet read on this forum.
- OMG..... our server but thanks for not destroying it.
I would have to ask ....

Wowee, the LAW finally caught me...

I don't have a lot of bandwidth to put into this, definitely not as much as Invitro, but perhaps to save you guys some work and to save our server we should be able to create a Top 40 or Top 100 users list on the website modeled after Invitro's approach or something similar, which would update once a week. Let me know if this is a good plan.

No. It is almost certainly more work than you think to duplicate this with the little tidbits like glitched-score ignoring. I say "no" because there are some things you could do that would help greatly for hopefully tiny bits of work! See below...

I am FROWNING on the method (that server runs some fairly critical operations) but I like the goal.

I don't like to overload servers. I was a webmaster / databasemaster / serveradmin for some years myself.

The main reason the method takes so long is that your webpage can only show 10 scores at a time. If it showed 100 at a time, it would take 10 times less time.

As far as an API goes, I'd have to ask, although I would definitely prefer dumping 1000 records once or twice per table to 4 hrs of scraping. Let me know if you want me to ask, CAUTION: conversation may go like this:
ME: Hey can we make an API or an export function so anyone can download our leaderboard data en masse?
VP: Um, why?
ME: Well... [insert explanation of data scraping and server load]
VP: Just make it so that doesn't happen.
ME: Erkay.....
Of course, he may also say "Sure, go for it". Say the word and I'll venture to ask about it.

The optimum solution for me would be to allow me to download the leaderboards as a single .csv file, through the web. No API needed and one export of the .csv daily, with the data being what they are now, *plus* the platform that the score was made on. (Many people including me think these rankings could be improved a good bit with platform data, which you can't get from the webpage.) Well, if you have the date/time the score was made/stored as a field, then that would potentially be useful too.

You could even just copy the raw database file to a web-accessible place if it didn't have anything sensitive... I'm sure there is a way to extract data from any non-encrypted database.

I would prefer that the data be available to everyone but if privacy is important you could slap a simple serverauth on the .csv URL and throw me (and certain other people) a username/passwd.

I hope there is a way to come to an easy solution that everyone likes. I will do what I can to continue these rankings up to but not including the point of making Farsight sore at me. I do thank you guys for not coming down on me or hiding the leaderboards. But I understand that you may want or need to stop the scraping of the data.

(I hope that is sensible... I don't have much time to post/edit now but wanted to reply ASAP.)

 

[invitro]

I keep checking this thread to see if Xbox One will ever get there. I'd like to see how some of my scores compare and it might make me play the game more.

Here is why XBox One is not included:

(a) It has its own leaderboards, which are not accessible from the webpage. I have no way to get the data even if I wanted to. (Even if I owned an XBox One and its edition of TPA, I would not copy its leaderboards by hand.)

(b) Even if the webpage had a separate presentation of XBox One scores that allowed download, I would probably not just merge those with the other platforms. Trigger warning: sensitive issue approaching . I do own an Xbox 360 with TPA and played that for a year or so before the PC edition. Tables were much easier on XBox 360 than PC. I'm too lazy to find my XBox 360 high scores right now or boot up that infernal machine, but I think they were two to ten times my PC high scores on most tables, with BK, Firepower, and Gorgar being near the 10x multiplier. So I am nervous that the Xbox One game is not comparable with the PC and other editions. (I think that some people have claimed that the phone/handheld versions are much harder than PC/PS4, but I don't know of any data to support this, and the tiny bit of data with platform that I know of, which is the last shutyertrap tournament, does not support it, in some sense. I have not tried to label the top 100 players by platform by using what they say their platform is in this forum because I'm worried about data fuzziness but also am too lazy to do that right now, although I may do it later/soon.)

I hope that explains my current position on XBox One, which definitely has potential of changing based on data access and/or good logical arguments .

[ Ad: I've signed up for the BlahCade March Tournament of the Month, have you? If not, click http://goo.gl/forms/HTjuvvolUK ]

 

[Mark Miwurdz]

No difference in gameplay between Xbox One and PS4. It's just which controller you prefer.

 

[Flipperdeflip]

(that server runs some fairly critical operations)

Are you sure you want the public to know that the leaderboard server is running fairly critical operations? That's like sending Failsight an invitation to DDOS that server.

 

[vikingerik]

I think that some people have claimed that the phone/handheld versions are much harder than PC/PS4, but I don't know of any data to support this

They're harder just because of the interface. A touchscreen just can't give consistent 1/60 second frame precision on flips and nudges as can a keyboard or gamepad. And a tiny phone screen is harder to see and make shots than a 24" monitor or 40" TV.

And phones have problems with battery and suspending and crashing during long games. So they'll be underrepresented on an all-time leaderboard compared to Blahcade's one-day tournament.

I don't know any reason the XBox game wouldn't be comparable with the PC. It's certainly possible you felt it was easier, but I'd sure bet that wasn't the platform itself, but some other difference in controller or hardware (display lag) or playing environment (couch vs desktop). Everything Miwurdz has been doing and showing off on the Xbox version is stuff I've been able to do on the PC as well. I'd like to see his scores included with the rest, although the problems with data collection are very understandable.

 

[EldarOfSuburbia]

Flipper/display lag and general reaction time are the issue with mobile platforms. Obviously a faster tablet/phone, and a controller, can make a lot of that go away, but I have difficulty with some shots on a tablet that I just don't see on PC.

Worst are long/fast orbit->upper flipper shots, such as Ripley's, STTNG, NGG. Those need very, very, precise timing to get right on my tablet. (TAF is better in that regard, but AdvX->left ramp is still harder on a tablet than on a PC.)

There's a reason I tend to prefer early-80s tables on my tablet, they play well and there's just not so much going on to tie up the processor. Tables like Gorgar, Space Shuttle, Firepower, F2K are great. The aforementioned RBION, STTNG, NGG, not so much.

 

[Mark Miwurdz]

They're harder just because of the interface. A touchscreen just can't give consistent 1/60 second frame precision on flips and nudges as can a keyboard or gamepad. And a tiny phone screen is harder to see and make shots than a 24" monitor or 40" TV.

And phones have problems with battery and suspending and crashing during long games. So they'll be underrepresented on an all-time leaderboard compared to Blahcade's one-day tournament.

I don't know any reason the XBox game wouldn't be comparable with the PC. It's certainly possible you felt it was easier, but I'd sure bet that wasn't the platform itself, but some other difference in controller or hardware (display lag) or playing environment (couch vs desktop). Everything Miwurdz has been doing and showing off on the Xbox version is stuff I've been able to do on the PC as well. I'd like to see his scores included with the rest, although the problems with data collection are very understandable.

I have posted some good scores recently on the PS4 version including a few #1 scores. I just don't have all the tables like I do on Xbox One.

 

[Slam23]

I think one major difference between platforms is how nudging is handled. I saw some PC footage and was amazed by how many presumably tiny nudges were allowed by the program without tilting. This gives a lot more (flipper) control. Also a touch scheme on mobile is never as fast/accurate as having two keys besides the flipper keys on the keyboard. I would still like to compete with the PC/console crowd on one general leaderboard like Invitro essentially created here but also like to see separate leaderboards for each individual platform.

@ElderofSuburbia: I do think my iPad Air 2 is a major step forward in processing power, I couldn't belief at first how smooth the game was when I upgraded from an iPad 2, and that was beside the great visuals on the retina resolution. I fired up my PC yesterday for Pillars of Eternity (which threatens to take away my pinball time ) and also did a quick round of Addam's Family in TPA. Maybe my PC is getting seriously outdated or I just have gotten used too much to my iPad, but gameflow was not that great. I experienced some serious flipper lag in any case, but then again, I may have to tune up my PC for TPA.

 

[LAW]

The optimum solution for me would be to allow me to download the leaderboards as a single .csv file, through the web.

Yep, this would be the most likely scenario, although it would probably be more like one csv per table. Once our overhaul to the leaderboards is complete, I'll see if this can be done without making waves on either side of the issue. The way the leaderboard system operates makes it difficult but not impossible to extrapolate the date and time of scores in tandem with how they rank in comparison to each other, and this may or may not be practical for 100,000+ records. Seeing how scores progress over time would be very interesting though.

*Note on xbox -we ourselves have no access to those scores (that i am aware of) :\

 

[jaredmorgs]

Yep, this would be the most likely scenario, although it would probably be more like one csv per table. Once our overhaul to the leaderboards is complete, I'll see if this can be done without making waves on either side of the issue. The way the leaderboard system operates makes it difficult but not impossible to extrapolate the date and time of scores in tandem with how they rank in comparison to each other, and this may or may not be practical for 100,000+ records. Seeing how scores progress over time would be very interesting though.

*Note on xbox -we ourselves have no access to those scores (that i am aware of) :\

While we're talking about leaderboards here, what do folks think about a regular leaderboard reset policy.

For example, every quarter?

Wouldn't it be great to see a constantly changing leaderboard instead of a long-standing Top 10 list?

Think of it like the operator setting the HSTD records to factory every 500 games in an arcade to encourage a new leaderboard chase.

 

[EldarOfSuburbia]

But we do get the Monthly/Weekly leaderboards anyway. Maybe make them more obvious?

 

[jaredmorgs]

But we do get the Monthly/Weekly leaderboards anyway. Maybe make them more obvious?

Yeah, that's a good point. So make those more of the "Default" and the All Time leaderboard as the historical record.

The problem I have is all the hacked scores in the All Time.

 

[Mark Miwurdz]

Yeah, that's a good point. So make those more of the "Default" and the All Time leaderboard as the historical record.

The problem I have is all the hacked scores in the All Time.

The solution is to keep on top of the hacked/glitched scores. It's that simple.

 

[Slam23]

+1 I wouldn't want to see legit scores go down every so often, if I have a monster game, I would like to have it stand for a looooong time That being said, the leaderboards could benefit from a good spring cleaning. And what's up with legit scores still getting posted to the wrong part of the leaderboards?

 

[xAzatothx]

While we're talking about leaderboards here, what do folks think about a regular leaderboard reset policy.

For example, every quarter?

Wouldn't it be great to see a constantly changing leaderboard instead of a long-standing Top 10 list?

Think of it like the operator setting the HSTD records to factory every 500 games in an arcade to encourage a new leaderboard chase.

Great idea. But think people are too precious about their scores and Farsight don't want to upset user base.

 

[jaredmorgs]

Great idea. But think people are too precious about their scores and Farsight don't want to upset user base.

I've kind of gathered that.

Oh well, the next best thing is to maintain them better. For the record, I would think that purging them would be easier than maintaining them from FarSight's perspective.

 

[Mark Miwurdz]

You put in a dozen hours to post a score... There's no reason that should be removed. This topic takes note of iffy scores (such as #1 Centaur, but not #1 ST TNG, unfortunately) so it's not so bad.

 

[Kratos3]

No difference in gameplay between Xbox One and PS4. It's just which controller you prefer.

Which do you prefer?

 

[invitro]

And what's up with legit scores still getting posted to the wrong part of the leaderboards?

Is this happening for a table other than TotAN? I don't remember evidence of it, but my memory has been a little shaky...

 

[invitro]

I've finally looked at all the Top 10 scores for more usernames & scores that should be blocked. I've blocked two usernames and three scores:

User 'klitschko' has these scores (note: "DATE" is approximate date of when score was recorded; the actual date is usually within this date and a week prior but may be up to a month prior; "ORANK" is the score's original rank):

# ID TABLE      |   RANK |                SCORE | DATE       |  ORANK | USERNAME             |
# 51 PotO       |      1 |        1,399,999,999 | 2015-02-20 |      1 | klitschko            |
#  9 Gorgar     |      2 |          999,999,999 | 2015-02-20 |      2 | klitschko            |
# 52 PZone      |     12 |          999,999,999 | 2015-02-20 |     10 | klitschko            |
#  7 FH         |     12 |          999,999,999 | 2015-02-20 |     12 | klitschko            |
# 24 CC         |     14 |          999,999,999 | 2015-01-17 |     12 | klitschko            |
# 12 HD3E       |     92 |          999,999,999 | 2015-02-20 |     88 | klitschko            |
# 45 HS         |   9644 |            1,468,800 | 2015-03-27 |   9609 | klitschko            |

That is more than enough evidence to block him.

Next, these scores are high enough to be blocked:

Theater => {
Tanico => 14_078_474_451_920, # added 2015-04-01
},
BigShot => {
perseus2273 => 193_278_500, # added 2015-04-01
},
Genie => {
HOIDA => 739_001_170, # added 2015-04-01
},

I don't know if these are hacks or glitches. Tanico and HOIDA have no other top-20,000 scores. perseus2273 has 22 other top-1000 scores (2 in top 20, 7 more top 500) but that Big Shot score is just too high.

Next, Starship Troopers has two strange scores (now, "DATE" = exact date I grabbed the score; "1ST-DATE" = same as "DATE" above; "RANK-COUNT" = # of scores in top 10/100/1000/10,000):

TRUE RANK USERNAME           |                 SCORE |       DATE |   1ST-DATE | RANK-COUNT  | BAD
   1    1 innomines          |       19,999,999,990  | 2015-04-01 | 2015-02-02 | 1/1/5/8     |
   2    2 Neronatusest       |        9,999,999,990  | 2015-04-01 | 2015-02-02 | 1/1/13/32   |
   3    3 Space Tractor      |        6,566,673,410  | 2015-04-01 | 2015-03-14 | 28/47/55/56 |
   4    4 _Mad               |        2,252,511,400  | 2015-04-01 | 2015-02-12 | 2/6/24/47   |
   5    5 bigdadwizard       |        1,203,034,960  | 2015-04-01 | 2015-02-02 | 1/1/1/8     |
   -    6 FailSight          |       [1,183,600,910] | 2015-04-01 | 2015-02-02 | 13/13/14/15 | hacker!
   6    7 Pop                |          986,178,120  | 2015-04-01 | 2015-02-02 | 1/2/7/16    |
   7    8 Sun Vulcan         |          833,481,070  | 2015-04-01 | 2015-02-02 | 2/11/43/53  |
   8    9 Vic Golf Chad      |          782,292,710  | 2015-04-01 | 2015-02-02 | 2/18/51/55  |
   9   10 TYH                |          761,001,970  | 2015-04-01 | 2015-02-02 | 1/1/12/24   |
  10   11 Toolinit           |          739,135,830  | 2015-04-01 | 2015-02-02 | 18/54/56/56 |

I haven't blocked them yet. I posted about them on the STroop forum. My guess would be that they are due to a ROM or TPA glitch/bug but I have no idea.

Now to Mr. Miwurdz' favorite, username 'www.vpforum.org's STTNG score of 1,890,199,057,550. I am finally going to block this user. Here are pros and cons to the legitimacy of this score:

pro: This user has 17 other top-100 scores. Perhaps all are hacks! Who knows?
pro: The score is less than twice as big as Mr. Miwurdz' #2 score.

con: I'm already blocking one of this user's scores: his #1 71,200,687,840 on BK2K.
con: He put up nine of his top-12 scores in a 4-day span between 2014-10-03 and 2014-10-07. This seems fishy. All his other scores are prior to 2014-02-25, when I started grabbing data.
con: I don't remember for sure but I seem to recall that a 1T score is impossible on STTNG, at least on some platforms.

I don't think it is crystal-clear that this user should be blocked, but the preponderance of evidence convinces me.