Requirements

Let's refine the design of an online code judge. At its heart, this system takes code submissions, runs them against predefined test cases, and determines if the submission is correct. However, building a robust, scalable, and maintainable judge requires careful consideration of various factors, specifically focusing on the code design. We want to simulate a real-world coding judge, taking into account compilation, execution, and result aggregation. The system should be able to support multiple programming languages and handle potentially malicious or resource-intensive code. Error messages should be as descriptive as possible to help debug. # Requirements ## Functional Requirements 1. **Code Submission:** * Accept code submissions from users in various programming languages (e.g., Java, Python, C++). * Store the code submission securely. * Associate each submission with a specific problem. * Limit submission size. 2. **Language Support:** * Support compilation and execution of code in multiple programming languages. * Provide a mechanism to easily add support for new languages. * Each language must have its own compilation and execution environment. * Consider versions for languages (e.g. Python 3.8 vs Python 3.10) 3. **Compilation:** * Compile the submitted code (if necessary, e.g., Java, C++). * Handle compilation errors gracefully and provide informative error messages to the user. * Impose time and memory limits on the compilation process. 4. **Execution:** * Execute the compiled code or interpreted code against a set of test cases. * Each test case consists of input data and expected output. * Capture the standard output (stdout) and standard error (stderr) of the executed code. * Enforce time and memory limits for each test case execution. * Provide the ability to define custom execution commands/flags for each language. 5. **Test Case Management:** * Define and manage test cases for each problem. * Support different types of test cases (e.g., input/output files, program arguments). * Allow associating different weights/scores for each test case. 6. **Result Evaluation:** * Compare the output of the executed code with the expected output for each test case. * Determine if the submission passed or failed each test case. * Calculate an overall score for the submission based on the test case results and weights. * Provide detailed feedback to the user, including: * Compilation errors (if any). * Runtime errors (if any). * Time and memory usage for each test case. * The input and expected output for failed test cases. * The actual output generated by the submission for failed test cases. 7. **Security:** * Prevent malicious code from compromising the system. * Implement appropriate sandboxing techniques to isolate the execution environment. * Limit access to system resources (e.g., file system, network). 8. **Resource Management:** * Manage system resources efficiently (CPU, memory, disk space). * Prevent submissions from consuming excessive resources and causing denial-of-service. * Implement resource limits on compilation and execution. ## Non-Functional Requirements 1. **Thread Safety:** * The system must be thread-safe to handle concurrent submissions from multiple users. Pay attention to shared resources and potential race conditions. 2. **Extensibility:** * The system should be designed to easily support new programming languages and test case types. * Adding a new language should not require significant code changes to the core judging logic. This suggests use of good object-oriented principles. 3. **Modularity:** * The system should be divided into well-defined modules with clear responsibilities. * Each module should be loosely coupled to other modules. 4. **Testability:** * The system should be designed to be easily testable. * Unit tests should be written for all core components. * Consider Dependency Injection to improve testability. 5. **Performance:** * The system should be able to handle a large number of submissions concurrently without significant performance degradation. Focus on algorithm efficiency. * Compilation and execution should be as fast as possible, within resource constraints. 6. **Error Handling:** * The system should handle errors gracefully and provide informative error messages to the user. * Log all errors and exceptions for debugging purposes. 7. **Security:** * The code judge must prevent execution of malicious code, protecting the underlying system. 8. **Maintainability:** * The system should be easy to understand, modify, and maintain. * Code should be well-documented and follow coding standards. ## Core Entities 1. **Submission:** Represents a user's code submission. Contains information such as: * Submission ID (unique identifier) * User ID (who submitted the code) * Problem ID (the problem the code is for) * Language (programming language used) * Code (the actual code submitted) * Submission Time * Status (e.g., "Pending," "Running," "Completed," "Accepted," "Wrong Answer," "Time Limit Exceeded," "Compilation Error") * Score 2. **Problem:** Represents a coding problem. Contains information such as: * Problem ID (unique identifier) * Title * Description * Test Cases (a collection of test cases) * Difficulty level * Time Limit (per test case) * Memory Limit (per test case) 3. **Test Case:** Represents a single test case for a problem. Contains information such as: * Test Case ID (unique identifier) * Input (the input data for the test case) * Expected Output (the expected output for the test case) * Weight (the score awarded if the test case passes) * Timeout (specific timeout for this test case, can override problem default) 4. **Language:** Represents a programming language supported by the judge. Contains information such as: * Language ID (unique identifier) * Name (e.g., "Java," "Python," "C++") * Version (e.g., "17", "3.9", "14") * Compilation Command (the command to compile the code) * Execution Command (the command to execute the code) * File Extension (e.g., ".java", ".py", ".cpp") 5. **ExecutionResult:** Represents the result of executing a submission against a single test case. Contains information such as: * Test Case ID * Submission ID * Status (e.g., "Passed," "Failed," "Time Limit Exceeded," "Runtime Error") * Actual Output (the output generated by the code) * Error Message (if any) * Execution Time (in milliseconds) * Memory Used (in bytes) 6. **Judge:** This is the core component responsible for orchestrating the compilation, execution, and evaluation of submissions. It likely utilizes strategy pattern to determine how to invoke compiler and executable based on the language. This could be implemented as an interface.

Design an online code judge.

Requirements

Think like an Architect

Premium Content