How judging is done
First, we compile your program. If it fails to compile, we judge it as Compile Error. Otherwise, we execute the compiled binary with the first test case. You can execute your program in a similar way by using the following shell command:
$ time ./your_program < sample.in > sample.out
If the execution takes too long, crashes or uses too much memory, we judge it as Time Limit Exceeded, Run Time Error, or Memory Limit Exceeded, respectively. While running the program we monitor its amount of output. If the program outputs much more data than we expect, we judge it as Output Limit Exceeded. If the execution completed without any error, we inspect the output produced to verify that it is correct. If the output is incorrect, we judge the program as Wrong Answer. When debugging, you can use the diff program to check if the output from your program matches the sample output. Note that for some problems, there can be many different acceptable answers.
$ diff sample.out sample.ans
If no error is found, we repeat this procedure with the next test case. As soon as an error is detected, we stop and report that error. Your program is executed anew for each test case, and time limits are per test case. We always apply the test cases in the same order.
In some cases, we provide extra information apart from the judgement itself, to help you debug your code. This information is available on the page for the respective submission (available by clicking on the corresponding submission ID in your list of submissions)
- Compile Error
- Run Time Error
- Time Limit Exceeded
- Wrong Answer
- Output Limit Exceeded
- Memory Limit Exceeded
- Judge Error
Accepted means that we were very happy with your program, and that it (as far as we could tell) solved the problem correctly. Congratulations!
Compile Error means that we failed to compile your source code. In order to help you debug the error, the compiler output is available as extra information. Information about what compilers and flags are used can be found on the Languages page.
Run Time Error means your program crashed during execution with our secret test input. More precisely it means that it terminated with a non-zero exit code, or with an uncaught exception.
Note that since the exit code is used to determine normal termination, it is important that your main function in a C or C++ program does not return a non-zero value.
The time limit for a problem is the maximum allowed running time on a single input set. For instance if there are three input sets, the time limit is two seconds, and your solution takes 1.94, 1.28, and 1.74 seconds on the three input sets, it does not get Time Limit Exceeded. But if it takes 0.01, 2.04, and 0.74 seconds on the three input sets, it does get Time Limit Exceeded (assuming the output of the program on the first input set is correct so that judging proceeds to the second input set).
We measure total CPU time used by your process, so you do not gain extra allotted running time by using multiple threads – it simply means that you reach the time limit faster.
When the time limit is exceeded, the program is terminated. There is no way for us to tell if your program was stuck in an infinite loop, if it was "just a bit" too slow, or somewhere in between. The output produced is not inspected until your program has finished successfully, so getting Time Limit Exceeded does not imply that the output you had produced so far was correct.
Note that, we run your submission on a wide range of test cases, typically much larger than the small examples provided in the problem statements. When debugging a Time Limit Exeeded it is helpful to consider the worst cases for your algorithm. Sometimes it takes a very carefully crafted test case to bring down a bad solution, and we try our best to find and include such cases.
Wrong Answer means that your program finished within the time limit, but that the answer produced was incorrect. This error is usually the most frustrating one, since typically no extra information is given. Sometimes, the only way around it is to try to find bugs in your code by constructing tricky test data yourself.
Note that wrong answer does not imply that your program is fast enough, or that it does not crash. Judging stops at the first error encountered, and your program may be too slow or crash on a later test case.
Output Limit Exceeded means that your program has produced too much output and we decided to shoot it down before it flooded our hard drive. Check to make sure that you don't get stuck in an infinite loop where you print something and that you handle input termination correctly.
Memory Limit Exceeded means that your program has tried to allocate more memory than the memory limit for the
problem. Note that trying to exceed the memory limit may sometimes result in other errors than MLE. An example
would be if you are using
malloc in C to allocate memory. If
because you are trying to allocate too much it simply returns a null pointer which, unless you check for it,
would probably cause you a Run Time Error when you try to use it. Similarly trying to allocate too much memory
in C++ using
new would cause a SIGABRT and give you Run Time Error.
Judge Error means that you've found a bug (or at least misconfiguration). Sorry! Please contact us.
We are currently using Dell PowerEdge R230 servers for judging. These are equipped with an Intel Xeon E3-1220V6 CPU running at 3.0 GHz and 8 GB RAM. A 64-bit Linux kernel is used.