We need new Moderators!
We are always on the lookout for talented people to join the team. That means you! If you think you could help us organise and inform the community while entertaining everyone then apply. We need people to help out on the forum, behind the scenes with announcements, on Discord and on our other Social Media channels.
If this is something you think might be of interest to you, HERE
Chance of Smithing Gems - Bug or Deliberate Inaccuracy?
Chris2307 (GB1)
Posts: 944
This is probably the most important thread I've ever posted in this forum and it regards the chance of smithing gems.
The chance of smithing a level 3 gem to level 4 gem is supposed to be 60% (http://prntscr.com/4zg3bs).
For a while, a number of players, including myself, have felt that this percentage chance was inaccurate. There was a thread about it a while ago and we all conceded that not enough evidence was present to accuse GGS of deliberately misleading us or of a bug (if we wish to assume GGS is innocent). So I have gone out of my way to record every single level 3 -> 4 smith attempt since then.
I have not mis-recorded a single smith; everything has been noted down. I've not asked any other player to smith a gem and report the result to me. Every single attempt has been from myself. Each smith has been fairly random and occurred whenever I had enough gems to attempt a smith. I recorded the results in an Excel spreadsheet and used its standard functions to calculate the percentages. The results are as follows:
http://prntscr.com/4zg1p6
I want a CM to comment on these results as quickly as possible. Is this an inaccuracy in your system or a deliberate act? Either way, this needs to be fixed. I present results consisting of 40 level 3 -> 4 attempts. This is enough to be statistically significant. You have no comeback whatsoever.
40 smith attempts: 47.5% success instead of 60% success as you advertise.
As a side note, I also include the results of my level 4 -> 5 attempts. Whilst I concede that the amount of smiths for this level are not enough to warrant an accusation based on statistically significant evidence, I believe that as I carry on, I will produce similar results with regards to the advertised chance.
I await your reply GGS...
The chance of smithing a level 3 gem to level 4 gem is supposed to be 60% (http://prntscr.com/4zg3bs).
For a while, a number of players, including myself, have felt that this percentage chance was inaccurate. There was a thread about it a while ago and we all conceded that not enough evidence was present to accuse GGS of deliberately misleading us or of a bug (if we wish to assume GGS is innocent). So I have gone out of my way to record every single level 3 -> 4 smith attempt since then.
I have not mis-recorded a single smith; everything has been noted down. I've not asked any other player to smith a gem and report the result to me. Every single attempt has been from myself. Each smith has been fairly random and occurred whenever I had enough gems to attempt a smith. I recorded the results in an Excel spreadsheet and used its standard functions to calculate the percentages. The results are as follows:
http://prntscr.com/4zg1p6
I want a CM to comment on these results as quickly as possible. Is this an inaccuracy in your system or a deliberate act? Either way, this needs to be fixed. I present results consisting of 40 level 3 -> 4 attempts. This is enough to be statistically significant. You have no comeback whatsoever.
40 smith attempts: 47.5% success instead of 60% success as you advertise.
As a side note, I also include the results of my level 4 -> 5 attempts. Whilst I concede that the amount of smiths for this level are not enough to warrant an accusation based on statistically significant evidence, I believe that as I carry on, I will produce similar results with regards to the advertised chance.
I await your reply GGS...
Post edited by Chris2307 (GB1) on
Chris2307 @ en 1
0
Comments
Don't click: https://goo.gl/IC6U01
Sadly, I share your fear but I do expect a reply from them considering I have provided evidence that we have been actively mislead in the game.
Of the House Of Dragons:
Order of the Shadow Dragons
Please show your maths if you want to make a claim like this.
I don't want to come off as patronising, but you haven't recorded enough examples to prove anything.
A quick back-of-a-matchbox calculation suggests that about 10% of people will see a similar (or greater) difference between the expected success rate and actual success rate (~5% of those seeing more failure, ~5% seeing more success). You might just have been unlucky.
It's also fair to say that those who see more success than expected don't come on the forum to complain, hence why the forum only contains anecdotal evidence of probabilities not matching up.
It's not to say that there isn't a bug, just that you don't have enough evidence to prove it, you would need to get at least 100 results showing a 47.5% success rate in order to do so without any doubt.
Spot on :P
We are the Battle Sacred Kittens
(for whom who really want to know the exact meaning of BSK, PM me ingame :P)
So basically, in a nutshell, you are saying that the gems have an independent probability. You are correct though, each forge is independent, but I do think that the success rate is inaccurate. more than half (50%) of mine fail.
Proud member of Praetorians
SO...I get conned out of rubies and support refuse to give them back. But when a ruby player does the same as me they get them back? This to me is point blank evidence that support and GGE don't give a fuck about us non ruby buyers...
http://prntscr.com/93xafb
Still think MeepMeep is one mad patty!
Actually, it is your explanation of statistics which is way off the mark here so let me explain, and of course it's percentage chance for each try - no one believes otherwise. In keeping with your 6 sided die example, I would not expect to roll the die 6 times and see each number. But over time, I would expect to see (and we would see) the results average out. With a coin flip - the same. Over X number of flips, we would see the flips average out over time. This is what I've attempted to do here. I have recorded 40 attempts and the overall result begins to average out. The more attempts there are, the more accurate the overall figure is.
I've actually showed my maths. You're issue is with the data set size and I take that on board.
100% agree with you here. This is why I've come to the forums with numbers to back up my claim. What I present is not anecdotal evidence anyway, although your opinion is that it's too small a number (40 attempts). I'll carry on recording anyhow. Please note that I had a "feeling" that the smithing chance was incorrect and I noted down every single smith in order to make this post very anti-anecdotal.
Again... your issue with my post seems to stem from the data-set size.
Fair enough - We'll see what we get when I go up to 100 then!
Yes and no. The numbers would average out, but not all the way. You are again confusing percent chance with statistics. Since I work at a casino, I see people who don't understand this all the time and refuse to get it. If you role one six-sided die, it is entirely possible to not get a six for 10 to 15 roles of the die. Does that mean that the chance of getting that six at one in six wrong even tho the percentages of actually getting it are closer to one in ten? I have also done some computer programing and my friends and I have done work with random number generators. If they use a base 10 system (step it by 10 percent) then it would be ten numbers they use. 1-6 would be success and 7 thru 10 would be failures. I have seen some of my programs come up with some very interesting results as far as the base ten system. This would be like using a ten sided die to get your numbers. The results would be even more interesting if they used a base 100 system (numbers 1 thru 100) to get an actual percentage number because then you would have both the tens and the ones place to worry about and there is "more" numbers that can show up in the "failure slot. The point is that the numbers you are using are not going to 100% match up to the percentage chance of success at the smithy.
Of the House Of Dragons:
Order of the Shadow Dragons
You're right about one thing. The results would never ever be 100% accurate and it's entirely possible that I could get 40 successful smiths in a row. However, you are still confusing statistics. I work as software development researcher so am very aware of both psuedo-random number generators and statistics. Whilst psuedo-random number generators are never truly random, they are random enough to not elicit the kind of inaccuracy I have presented thus-far. In fact, if you've programmed your programs correctly and have used proper built in RNGs instead of attempting to code your own, then you should not see "interesting" results from a random number generator at all. They do the job just fine. Anyway, the thread is digressing a bit.
Back on to the statistics. By increasing the data-set, the percentage results average out further and further getting closer to the correct result. Whilst never achieving a purely accurate result, it is the job of statistics to try and explain these numbers. I've run a basic binomial test here which is a standard way of testing this kind of thing (You can Google it if you like). If you run my findings through any Binomial Probability calculator, you will find that the binomial probability of the results I have achieved against the dataset I have collected is good enough to report as evidence.
I.e. Binomial Probability of the above = 0.03518. This is the same as saying that the chance of observing 19 successful smith attempts in 40 (based on a posted 60% success rate) is 3.518%. Anything less than 5% is considered to be statistically significant and therefore reliable.
Despite this, I'll continue collecting to further strengthen my hypothesis.
Advocate of speaking up regarding mental health and seeking help
Of the House Of Dragons:
Order of the Shadow Dragons
α ЯTFM ¿¿¿Want Free $tuff??? Then Go Write Santa a Letter ЯTFM Ω
I understand you work at a casino and I really mean no disrespect by this but it by no means gives you even a basic understanding of fundamental statistics (you'd need to take a class or read a proper book on it). We're going around in circles here and you're unable to explain yourself without resulting back to your experience at work; which I am sure is a good enough explanation to your customers. As a side note, I am not a customer to any casino (I have more important things to spend my money on).
You keep mentioning the example of a slots machine. Whilst this is fine and you're right; each individual spin is an independent chance, what you're failing to take in to account is the number of possible outcomes. With each additional outcome, the required dataset size increases. So right away you are comparing a slot machine with 3, 4 or possibly more outcomes against my test which has one of two outcomes. I've been able to show that 40 tests were enough (just) by presenting my p value. Again, maybe you should read up on this (as well as Binomial Testing - a statistically sound way to test for outcomes).
You've already said this. You're not thinking of the bigger picture. There is a way to test whether a system gives a fair (or expected outcome) by testing multiple trials. This is called binomial testing. I'm not testing each trial. (One of many good explanations here: http://stattrek.com/online-calculator/binomial.aspx#experiment) Again, look at my P value to satisfy yourself that my dataset is large enough.
Now you're sounding like a casino customer
Actually, no. Since I am not testing the behaviour of people (or something which is affected by the behaviour of people), I do not need to test using multiple persons (like you would in psychology). I am testing the overal behaviour of the smithing system and whether it produces fair results. Therefore I must test this aspect over and over and over again. So whilst I agree with you; the more data the better, I am afraid you are completely wrong by stating that more than one person must carry out the same test.
So I repeat to you, spend a couple of hours reading up about binomial testing and what it does. I didn't want to get in to this kind of debate on this forum as I want to focus on the results and what it means to my original questions; is GGS deliberately misleading us or is there a bug?
And please also remember;
You're experience/examples revolve around slot machines where many many outcomes can occur (i.e. Mini games on the modern style machines, different levels of payouts, different inputted amounts) which is much more complex and therefore requires much, much more testing.
I am testing one of two outcomes. Much simpler and I have used a statistically sound method. Please do not compare the two again. You're comparing apples and oranges.
I'm afraid you're mixing up the probability of a single outcome and the probability of a stated rate of success being flawed.
As a quick, stupid, example, use your binomial calculator to toss a fair coin 1000 times (heads=0.5, tails=0.5)
The probability of getting exactly 500 heads is 2.5%. It does not mean that the coin is unfair. It means that the probability of one outcome of one trial (trial: flip a coin 1000 times; outcome: observe exactly 500 heads) is low. It's improbable, but doesn't suggest that the coin is unfair, indeed, any other single outcome would have an even lower probability. It is on the contrary, strong evidence that the coin is fair - we observed the theoretical mean.
What you need to do, is set up a hypothesis assuming that everything is ok, and then show that it is sufficiently unlikely for that hypothesis to be true (I'm pretty sure you have come across hypothesis testing before as you talked about the 5% confidence interval). This has to be in one direction or the other or both.
Hypothesis: "The actual probability of upgrading a gem from 3 -> 4 is 60%"
The probability that given this hypothesis, and 40 trials with a stated success rate of p=0.6 , you observe 19 or fewer successes is 7.43% This is low, but not low enough to reject the hypothesis with a confidence interval of 5%.
Imo it is not enough to simply continue your experiment with your existing data - if you want to conduct a statistical test, you should decide the parameters of the experiment in advance - in this case, the number of trials. If you continue adding results to your data set from now, you are not calculating the probability of 50 trials being lower than a significant threshold, but the probability of 10 trials pushing your mean from 40 trials lower than it is now, which then becomes more likely, but ultimately meaningless when it comes to proving anything.
Okay, that's a fair point. I've reported the P(X = x) result and you are right; I need to report when P(X <= x) Again, this comes down to the low number of recorded trials. Whilst not at a significant level, I still believe my hypothesis (that is "The actual probability of upgrading a gem from 3 -> 4 is less than 60%) and I've still reported a low confidence interval, which I believe is an indicator that this is worth further investigation.
However, I still see no reason why the original dataset cannot be used. As far as I can see, I have reported my results too soon (if my hypothesis is true) and adding to the original data recorded is OK considering that nothing has changed since reporting the data here. I have continued recording and still have not missed a single outcome. I.e. There would be no difference whether I started collecting data again now or adding on as my method has not changed.
Anyway, thank you for pointing out the confusion between the P(X = x) and P(X <= x) results. I still stand by my previous points in regard to my conversation with Colpin Cathar as he was arguing a different point.
I know you believe it, which is why there is a lot of scope for bias (statistical or otherwise) if you don't run the experiment properly.
Maybe you can use it, maybe you can't - to be completely honest, it has been a long time since I had to do this kind of testing. But you may want to inquire a bit more with people more knowledgeable than myself - I strongly suspect that if you continue adding data to your data set, you will end up with a flawed experiment.
A relevant quote from a statistics blog: http://doingbayesiandataanalysis.blogspot.de/2013/11/optional-stopping-in-data-collection-p.html
Reading up on prior/posterior probabilities, part of Bayesian probability, might help too - but personally, this wasn't my strong point.
Ideally all your experiment design decisions should be made before any data collection, and more importantly, before any data analysis, i.e. clear criteria for accepting/rejecting a hypothesis, a clear pre-defined stopping point to an experiment, and any possible extensions (again pre-defined, before any data analysis).
Thanks for the link. I'll certainly have a read up on this.
When we talk about bias, this is something which I am, of course, aware of and I've spent a lot of time in the past accounting for bias in other experiments (Completely different experiments this one) so I was aware of this going in. I struggle to account for any potential experimenter bias and the data collected thus far has adhered to some basic criteria in that I have recorded 100% of the results. I think that although you are right about defining all experiment design decisions before going in, I think I will be OK to carry on with this particular data collection.
Either way, I appreciate the information you've given me.
I can tell you from experience that there are some people who come in, sit down at a machine and will not move for 3 days, and not hit a "jackpot". Then there are those who come in, sit down, and within 3 hours, they have won 5 of them. It is all random. If it the machine is going to hit, then it will hit, if it's not, well, then it won't. In a way, the idea is similar. A lot of players don't come in just to play. The want the "jackpot". If you look at the system in that respect, either "jackpot" or "no jackpot" then it is only two outcomes.
Of the House Of Dragons:
Order of the Shadow Dragons
I.. I want a jackpot.
Advocate of speaking up regarding mental health and seeking help