About AI-Therapy

AI-Therapy creates online fully automated programs using the latest evidence-based treatments, such as cognitive behavioural therapy. The social anxiety program has been tested with a randomized controlled trial. To find out more visit:

These results were generate by Comparing two sets of data on November 11, 2023 at 10:23 AM (UTC).

Results

Data set overview

If you would like to share these results, please use the following link:

https://www.ai-therapy.com/psychology-statistics/results/20231111102320935
Data set namePython Code Correctness
Data set notes(not provided)

Data set statistics

Sample nameNumber of samplesMeanStandard error of the meanStandard deviationMedian
Copilot6000.3750.0200.4850.000
ChatGPT6000.2080.0170.4060.000

Test results

Number of samplesN = 1200
Wilcoxon rank-sum
  • W = 330300.000
  • z-score = -4.998
  • Asymptotic significance (2-tailed) p < 0.001
  • Based on a significance level of 0.01, there is a statistically siginificant difference between 'Copilot' and 'ChatGPT'.
  • Effect size r = -0.144
Mann-Whitney
  • U = 150000.000
  • z-score = -6.348
  • Asymptotic significance (2-tailed) p < 0.001
  • Based on a significance level of 0.01, there is a statistically siginificant difference between 'Copilot' and 'ChatGPT'.
  • Effect size r = -0.183

Visualization

The graph above is a bihistogram that allows you to compare the distribution of Copilot (top) with the distribution of ChatGPT (bottom).