I run various A/B Split tests online. I always set all my tests for 10,000 visitors so page A and B are each equally rotated 5,000 times. My problem is that 10,000 is just an arbitrary number and doesn't really give me the true sample size that I should run, so in some tests I may have a good sample at 2,000 and in other tests I may need to go up to 20,000 pages to get an accurate result, but I have no idea. Is there a formula that I can use to determine the sample size in real-time so that as data is collected, the approximate sample size is calculated? Here's a real example based on data from two real-life tests that I'm running right now as I write this...
Test 1: A and B both have 4616 page views each. A has 82 conversions for a 1.78% conversion rate, and B has 76 conversions for a 1.65% conversion rate. This test is almost about to end, but should it? What should the sample size really be to get an accurate answer?
Test 2: A and B both have 1532 page views each. A has 50 conversions for a 3.26% conversion rate, and B has 36 conversions for a 2.35% conversion rate. Based on my 10,000 page limit, This test still has a while to go, but A already has a large 38.89% lead over B. What should the sample size really be to get an accurate answer?