don't click here

Validity of proof and clarification


    Print

Author Topic: Validity of proof and clarification  (Read 9009 times)

0 Members and 1 Guest are viewing this topic.

Offline TimpZ

Validity of proof and clarification
« on: May 27, 2015, 08:54:50 am »
Before I start, no this is not the new "emulators should be banned" thread.

Today I finished a run of Sonic 2 using savestates and uploaded to Youtube. If I was a douchebag who wanted to cheat I could reference that video when submitting stats and no one could BS call me because nothing in the video indicates cheating.

Except it's completely ridiculous.

No one in their right mind who had played Sonic 2 for more than 10 minutes would think this run was made at least without using some form of cheat. The entire speedrun community would be unanimous in thinking that the run isn't legit.

TSC operates on an honour system and if someone comes under suspicion of cheating they are required to provide proof. This proof is then analyzed by moderators and whomever they might ask for help. I have in the past helped out a number of times in trying to help ban people who are under suspicion. This includes audio, video and gameplay analysis. After I have determined the most likely emulator the game is being played on (if not console) by using telltale signs, I re-watch the video a few times, sometimes slowed down, to try to detect frame skips to look for a splice, unlikely frame perfect inputs (like counting the number of frames some suspicious spindashes take) and a number of other things that might differ depending on the emulator. This is all made harder by YT compressed, 30fps, low quality recordings I might add. If everything checks out and there's several stats submitted, you can look at other videos the person has submitted and do the same thing. All this work can take hours if not a whole day put together.

If that checks out, the only thing left to do is to analyse what the person is actually doing during the run and weigh it all together to see if it's reasonable. For IL's this is very hard almost to the point of pointless but sometimes someone uploads a supposed single segment full game run and submits the times in that run to TSC or other sites.

This is an example of when this was successful in detecting someone cheating who later got caught from another video and admitted it. He denied that the full game run he used for submission was cheated but his credibility was already lost. The community of different games have also used this form of analysing gameplay to determine whether a run is in fact "too good to be true", most notably on SDA.

I've been told that an SDA reject for cheating can be valid proof for a BS call, but this does not apply to emulators. It could also probably be extended to other sites like speedrun.com, but what happens if the player only submits to TSC?

A few months ago I sent this to an admin who requested my help who then forwarded to a second admin. I admit that it's quite passive aggressive to the point of not being an objective analysis anymore and while I don't want to put full blame on frustration I think it's important to note that I've tried to help a growing number of times fruitlessly and this BS call in particular took an unusual amount of time. After a month of silence I requested response but still I have received none whatsoever. The video in question has since been taken down after being made private shortly after it caught knowledgeable peoples attention but I have a download of it saved on my computer. The few other prominent runners I have asked all agree it looks mighty suspicious with just a quick glance. The stats are still up.


So my question is: How rigid does proof need to be in order to be considered valid for a BS call? Would my video really make the cut? At what point can we trust the knowledge of top runners without obvious proof? What can we do to prevent this loop hole to continue to happen? Hell, this guy is not banned but admitted to cheating just months prior using exactly the same setup I did in my video. This is a big issue to me with this site and I think a clear line has to be drawn somewhere.
« Last Edit: May 27, 2015, 09:03:22 am by TimpZ »

Offline Starlight_Glimmer

Re: Validity of proof and clarification
« Reply #1 on: May 29, 2015, 01:38:46 am »
man, idk what to tell you, I see your agruement, I understand it completely.

I know nothing about sonic 2 an it looks legit to me.

I guess the best way to describe why theres a lack of people caring is because most people are saying "it doesnt effect me, so idc." Which I had the same attitude about not to long ago until I learned about how many record's this is actually screwing me out of/ screwing the whole site over on.

its a difficult topic to explain, but once you know... its like just sitting and watching tsc get raped every day and you want to stop it, you want to alert everyone, but you cant, and its painful to watch.

im extremely optimistic about tsc though, I say let these kids run wild with their "legit stats" there will be a breaking point eventually, which... unfortunately might be so long from now it will be a site nuke, but so be it.

anyone that thinks we can just have an honor system on the internet in the year 2015.... is ... *suppresses extreme mean words* an idiot.
We Shine Like Rainbows~ 

Offline TimpZ

Re: Validity of proof and clarification
« Reply #2 on: May 29, 2015, 11:55:26 am »
I'm not saying TSC shouldn't have an honour system or that every stat should have proof, but if easily faked proof is accepted then BS calls become ineffective and pointless, leaving the site with no method of weeding out fake stats. What is valid proof varies with the game, but I'm going to use Sonic 2 as an example since I know a lot about it.

We can make up a few types of evidence and order them after how hard they are to fake:
- Claim
- Screenshot/photo
- Video
- Live attempt

The problem in this case is that TSC accept all videos as valid evidence for any stat and if someone has a problem with it they need to find proof of it being faked. However all videos are not equally hard to fake which creates a discrepancy in the order I laid out. We can try to make a new one with more types:

- Claim
- Screenshot/Photo
- Emulator video with capability of recording inputs
- Emulator video without capability of recording inputs
- Emulator video with extra info (e.g. including booting the game or controller cam)
- Console video
- Live attempt

Now we need to decide how to handle these different types of evidence. Claims you do when you submit stats so that's nothing new. Photo's shouldn't be valid proof because how easy it is to fake thanks to the debug mode allowing you to move with a paused timer. Console videos and live attempts could probably be considered valid by default since faking them is very hard and detecting splices or similar is much easier. But if someone submits an emulator video I think additional information should be required if a BS call comes into effect (like sharing the GMV if it the inputs were recorded, sharing the source video files etc.) and I think it should be viewed with a lot of skepticism similar to screenshots instead of being unfalsifiable proof like it is today. You could also restrict what methods one is allowed to use or have clear instructions that makes cheating harder while also making detecting straying from these easier. I've heard that people are working on saving meta-data in the input files in Bizhawk that will make cheating on it very hard so I'm hopeful reasonable alternatives for providing proof can exist for everyone.
« Last Edit: May 29, 2015, 12:26:18 pm by TimpZ »

Offline Starlight_Glimmer

Re: Validity of proof and clarification
« Reply #3 on: May 30, 2015, 01:02:16 am »
Yah, we're on the same team. with some slight differences. I have never believed in an honor system, im pro bsing everything with even a slight whiff of bs (which, if you think about it, so are you; if you look into something, in any way, you're calling bs on a personal level), friends irl can/will screw you over... so I think its absurd to think that people on the internet won't.

I fucked up bad, and I pretty much screwed a large portion of tsc ( this part hurts the most) when I went all out on emulators. I was for sure TSC was going to ban them, It was close, but they didn't, but what I left now was an exposed hand of what I/we are looking for. I had to, to try and bring novices and experts alike up to speed, but now they just know what to try to hide/honor system out on.

Now you took it a step forward, and for anyone that knows sonic 2 you taught them how to cheat in sonic 2 with our favorite console, the emulator. I"m not bashing you on this. This is what all the good guys are going to have to do now. you have to try to purposely abuse emulators, record (not submit >_> im not encouraging cheating), share, and explain them to everyone. (ala dreamcast emulator ban)

I know this topic isn't about emulators, but, if we are taking baby steps forward, this would be a huge step in overcoming a lot of proof issues. its one thing to check emulator abuse, its... somewhat easier to check emulator vs console use.
We Shine Like Rainbows~ 

Offline FocusSight64

  • Language Geek
  • TSC Profile
  • win10 firefox
  • Posts: 190
  • Your Friendly Neighbourhood Weeaboo
    • View Profile
Re: Validity of proof and clarification
« Reply #4 on: May 30, 2015, 03:05:28 am »
By "console video", I assume you mean using a capture card to actual record from the console. It'd be awful to record from a camera to show that you are indeed playing on a console if you already have a capture card. Anyways, I do think it's good that people on the site are actively thinking on adding a system of proof that is incorporated into the site itself. It's something TSC has desperately needed for years.
Proud supporter of Cookie☆ (クッキー☆) MADs.


Offline TimpZ

Re: Validity of proof and clarification
« Reply #5 on: May 30, 2015, 07:33:32 am »
By console video I mean footage captured from a console. Everyone and their mom have a decent camera on their phone nowadays so there's no excuse not being able to record in the case of having to prove a stat.

    Print
 

Hits: 209 | Hits This Month: 1 | DB Calls: 8 | Mem Usage: 1.01 MB | Time: 0.05s | Printable

The Sonic Center v3.9
Copyright 2003-2011 by The Sonic Center Team.