Discovery Document: Integrating Security Suites into GitHub CI for Open edX

Owned by Maga Jaimes
Mar 20, 2024
10 min read

Summary

This document presents the findings and recommendations of the Security Working Group on integrating security suites into the GitHub Continuous Integration (CI) system for Open edX. Aligned with OEP-60's focus on proactive security, the document evaluates various security tools, considering their potential integration into GitHub CI for Open edX. It explores tools across Static Application Security Testing (SAST), Dynamic Application Security Testing (DAST), Software Composition Analysis (SCA), and Secrets Detection. The aim is to bolster the project's security posture by providing the community with detailed insights into the capabilities of each tool, thereby facilitating informed decision-making.

Background

As an evolving open-source learning platform, Open edX faces a range of specific security challenges. These include ensuring the integrity and security of its code in a collaborative development environment, protecting sensitive user data, and maintaining robust defenses against external threats and vulnerabilities. Embedding effective security measures within the development pipeline is essential to safeguard the platform and its users.

Open edX currently utilizes Dependabot for monitoring and updating dependencies in its main branches, complemented by select features of GitHub Advanced Security at the organizational level. However, the GitHub CI environment lacks a comprehensive suite of tools capable of performing broader security checks. Integrating advanced security tools into the GitHub CI pipeline is critical for enhancing the organization’s overall security posture, especially before deploying code to production.

In alignment with OEP-60’s strategy for proactive security enhancements, this initiative explores the integration of advanced security suites into Open edX’s GitHub CI workflows. This proactive approach aims to strengthen the security framework of the platform, addressing its specific challenges and safeguarding it against the evolving landscape of digital threats.

Overview of Security Tool Categories

In this document, the focus is on evaluating tools that fall into four primary categories essential to DevSecOps practices. Below is an overview of these categories, providing context for the types of tools considered:

Static Application Security Testing (SAST) Tools

  • Function: Analyze the source code without executing it to identify potential security vulnerabilities.

  • Purpose: Detecting security flaws within the code before the application is run or deployed.

Dynamic Application Security Testing (DAST) Tools

  • Function: DAST tools test the application in its live, running state, simulating external attacks to uncover security weaknesses.

  • Purpose: Identifying runtime vulnerabilities and weaknesses in user authentication or server configurations.

Software Composition Analysis (SCA) Tools

  • Function: Focus on analyzing external components and libraries used in an application, including open-source code.

  • Purpose: They ensure that these components are free from known security vulnerabilities, up-to-date, and compliant with licensing requirements.

  • Note: Since Dependabot is already set up to monitor our main branches, our discussion on SCA tools in this document will be minimal.

Evaluation Methodology

The following aspects were taken into consideration when evaluating the possible tools to integrate into the GitHub CI:

Key Evaluation Criteria

  1. Security Capabilities: Assessed the specific security functionalities of each tool, ensuring they align with Open edX’s requirements.

  2. Ease of Integration with GitHub CI: Looked at how simply each tool could be integrated into our existing GitHub CI setup, seeking solutions that would blend easily.

  3. Compatibility with Open edX’s Tech Stack: Tools needed to work effectively with languages like Python, and Javascript, and frameworks like Django, and React, which are core to Open edX.

  4. Performance Impact: Evaluated the effect of each tool on the speed and efficiency of the development and CI/CD process, aiming to maintain optimal performance.

  5. User-Friendly Design: Prioritized tools that are straightforward to set up and use, facilitating ease of adoption within the community.

  6. Cost-Effectiveness: As an open-source project, the cost and value proposition of each tool were significant factors, with a preference for free viable options.

  7. Community and Documentation Support: Preferred tools with robust community backing and comprehensive documentation, aiding long-term support and troubleshooting.

  8. Emphasis on Security Over Code Quality: Focused on tools that prioritize security enhancements over mere code quality improvements.

Tool Evaluation

1. CodeQL (SAST)

2. FluidAttacks (SAST)

  • Overview: FluidAttacks is an open-source Static Application Security Testing (SAST) scanner designed to identify security vulnerabilities within web applications. Leveraging static analysis and pattern-matching techniques, it efficiently scans applications across various programming languages, including Python, and JavaScript, and supports frameworks such as Django and React.

  • Integration: Offers straightforward integration with GitHub CI pipelines through Github Actions .

  • Key Features:

    • Multi-Language Support and Framework Compatibility: Capable of scanning applications written in Python, JavaScript, and more, making it versatile for projects across different technology stacks.

    • Detailed Reporting: Generates comprehensive reports in both pretty-printed and CSV formats, detailing identified vulnerabilities with recommendations for remediation, aiding developers in prioritizing and addressing security concerns.

  • Community Support: As an open-source project, FluidAttacks benefits from the collaborative efforts of a community of developers and security professionals, ensuring regular updates and continuous improvement of the tool.

  • Pricing: Freely available, offering its advanced SAST capabilities at no cost, aligning with its mission to make application security accessible to a wider audience.

  • Links:

3. Bandit (SAST)

  • Overview: Bandit is a Python package designed specifically for analyzing Python applications to identify common security issues. After scanning the files, it compiles a detailed report of its findings, offering insights into potential security vulnerabilities within the codebase.

  • Integration: Configuring Bandit is straightforward, as it utilizes a YAML file for its settings, making it easy to integrate into GitHub Actions workflows.

  • Key Features: Checks for security issues like SQL injection, and password mismanagement.

  • Performance Impact: Lightweight, minimal impact on build times.

  • Ease of Use: Straightforward to set up and configure for Python projects.

  • Community Support: Good community backing, part of PyCQA.

  • Considerations: Focused on Python only.

  • Pricing: Free and open-source.

  • Links:

4. Django Security Checks (SAST)

  • Overview: Django's check --deploy command executes a series of predefined checks, offering an automated and straightforward method to enhance a site's security posture.

  • Key Features:

    • Django-Focused Security: Directly targets security aspects relevant to Django applications.

    • Automated Checks: Facilitates automated security checks within the GitHub CI environment.

    • Customizable Integration: Flexible integration through existing or custom GitHub Actions.

  • Considerations:

    • These security checks do not guarantee a secure site. They are not designed to audit code, detect intrusions, or perform complex security assessments. Instead, they offer an automated way to review common security oversights, acting as a preliminary step to bolster site security.

    • Additionally, given that some checks may not be relevant or applicable depending on the deployment configuration, adjustments may be necessary to ensure the command's effectiveness.

  • Integration: The process involves executing a command, making it straightforward to integrate into a GitHub Action workflow. This ease of integration provides a quick way to incorporate Django's security checks into continuous integration pipelines

  • Pricing: Free.

  • Links:

5. ESLint Security Plugin (SAST)

  • Overview: The ESLint security plugin, often referred to as eslint-plugin-security, helps identify problematic patterns in JavaScript code. This plugin specifically focuses on identifying code that may have security vulnerabilities or bad practices that could lead to security issues.

  • Integration: This plugin can be effortlessly integrated into GitHub CI workflows by executing specific commands within a GitHub Actions workflow file. Both ESLint and the ESLint Security Plugin can be directly set up within a GitHub Actions workflow, eliminating the need for individual repository installations.

  • Key Features:

    • Detects Potential Vulnerabilities: Includes a set of rules designed to highlight potential security issues such as those related to code injection, data leakage, or other common web vulnerabilities.

    • Customizable Rules: Like ESLint, the security plugin allows you to enable, disable, or configure the severity of individual rules to fit your project's needs.

  • Performance Impact: The plugin is known for its speed and efficiency, making it particularly beneficial for projects with extensive JavaScript codebases.

  • Considerations: As the ESLint Security Plugin is a critical dependency for identifying potential security vulnerabilities, it's important to regularly update it to the latest stable version.

  • Ease of Use: Easy to configure and run the commands, especially for JavaScript developers.

  • Community Support: It benefits from a broad user base and is extensively utilized across numerous JavaScript projects, ensuring robust community support.

  • Pricing: The plugin is open-source and available at no cost.

  • Links:

6. OWASP ZAP (DAST)

  • Overview: OWASP ZAP (Zed Attack Proxy) is an open-source Dynamic Application Security Testing tool designed to identify vulnerabilities in web applications. It stands out for its ability to automatically find security flaws in web apps, making it an important tool for developers and security teams aiming to improve their web application security posture.

  • Integration: ZAP's GitHub Actions are available on the GitHub Marketplace, offering a range of actions for different security testing needs, and facilitating the integration of automated security testing into CI/CD pipelines.

  • Key Features: Automated penetration testing, active community development.

  • Performance Impact: Flexible, can be tailored to perform quick scans for immediate feedback or more thorough, in-depth testing, depending on the project's needs.

  • Community Support: Enjoys strong support from the OWASP community and the broader security community. It is recognized as one of the most widely used tools worldwide, and it is praised for its robust community engagement and comprehensive security testing capabilities.

  • Considerations:

    • Mitigating Impact on Services: Scans performed by the tool can sometimes have unintended side effects on the target website or application. For instance, ZAP's interactions with web forms can lead to an overwhelming number of submissions, which might be misconstrued as spam or even a denial-of-service attack. Therefore, the scan should be planned to minimize disruptive impacts. This could involve scheduling tests during off-peak hours or configuring ZAP to exclude certain actions or parts of the site.

    • Informing Service Providers: In addition to securing permission from the website or application owners, it's important to communicate with any third-party services that might be affected by the scanning. This includes hosting providers, CDNs, and other infrastructure services. These entities should be informed about the planned testing to prevent any misunderstanding or unintended consequences. For example, a security scan could inadvertently trigger a defensive response from a CDN, mistaking the scan for a genuine attack.

    • Learning Curve: The integration and configuration of the tool, especially for individuals new to DAST or automated security testing tools, can present a learning curve.

  • Pricing: Free and open-source.

  • Links:

7. Socket (SCA)

  • Overview: Socket is an open-source Software Composition Analysis tool that protects codebases from software supply chain attacks by analyzing dependencies. It specializes in identifying and evaluating the security risk of new dependencies introduced in pull requests. It focuses primarily on NPM dependencies within the JavaScript ecosystem.

  • Integration: Socket integrates as a GitHub app. It automatically monitors pull requests for changes to package manifest files such as package.json, package-lock.json, and yarn.lock for NPM, as well as corresponding files in the PyPi ecosystem. When a new dependency is added, Socket analyzes its behavior and security risk, providing feedback directly in the pull request.

  • Key Features:

    • Proactive Supply Chain Protection: Warns developers about risky dependencies added to pull requests.

    • Content-based Analysis: Identifies malicious code and risky behaviors in dependencies through static analysis.

  • Performance Impact: Designed to be flexible, allowing for quick scans or thorough analysis based on the project's needs. Its integration and operation do not significantly impede the development workflow.

  • Ease of Use: ts integration into GitHub simplifies the setup process, making it accessible for developers to start using with minimal configuration.

  • Community Support: Backed by strong support from both the security and open-source communities, Socket is recognized for its valuable contribution to software security.

  • Pricing: Offers a free tier for open-source projects, which includes unlimited public repositories.

  • Links:

 

Factors to consider before selecting the final tools

Risks

When integrating new tools into the GitHub CI pipeline for Open edX, there are potential risks that should be carefully considered:

  • Performance Overhead: Tools that significantly slow down the CI/CD process should be approached with caution. For projects like Open edX, with frequent updates, it's essential to maintain efficient build and deployment times.

  • Overlap in Features: The presence of feature overlap among tools, particularly in areas like SCA or SAST, could lead to redundancy. It’s important to strategically select tools to avoid unnecessary complexity in the security framework.

  • Language/Framework Specific Limitations: The compatibility and effectiveness of each tool with the programming languages, frameworks, and technologies used in Open edX are essential considerations. Tools must be chosen based on their ability to accurately analyze and provide insights relevant to the project’s specific technological stack.

  • Cost and Licensing

Layered Security Approach

Using a layered security approach when picking security tools, by using a mix of security tools like SAST, DAST, and SCA, would help us create a more complete and effective defense strategy, and help us reduce the risk of problems and security incidents in various parts of our software.

References and Resources

Next Steps

  • Selecting Tools for Pilot Tests:

    • Objective: To determine the efficacy of each security tool in the context of Open edX's environment and workflow.

    • Action Plan: From the feedback garnered in this document, select tools for pilot testing. Conduct these tests to evaluate each tool's integration capabilities and effectiveness within the Open edX ecosystem, thereby enabling informed decision-making based on performance and fit.

  • Conducting Pilot Tests and Establishing Protocols:

    • Objective: To assess how each security tool performs in the specific context of Open edX’s environment and workflow.

    • Action Plan: Conduct a trial or pilot phase for each selected tool, where its integration capabilities and effectiveness within the Open edX environment are evaluated. It’s also necessary to establish a protocol for regularly reviewing and acting on the findings and recommendations provided by these tools, ensuring that all identified issues are promptly addressed. This approach will facilitate making informed decisions based on the tool’s performance and suitability.

  • Optimizing GitHub Advanced Security.

    • Objective: To maximize the utility of GitHub Advanced Security features within the organization , focusing on GitHub secret scanning and Dependabot to prevent secret leaks in the code and ensure dependencies are updated to the latest secure versions.

    • Action Plan: Ensure that the GitHub Advanced Security features are enabled and being used effectively. Review the current usage of these features to identify any gaps or areas for improvement, ensuring that the organization is leveraging the full potential of these security measures.

  • Securing GitHub CI Pipelines

    • Objective: To safeguard GitHub CI pipelines against potential security threats.

    • Action Plan:

      • Investigation: Conduct a thorough investigation of the current GitHub CI pipelines to identify any security vulnerabilities.

      • Documentation: Develop a comprehensive document outlining minimum security requirements and best practices for securing GitHub CI workflows within the organization.

      • Tool Evaluation: Explore and evaluate any existing tools that could improve the security of GitHub CI pipelines.

      • Some links that might be helpful:

This document is subject to updates and revisions as the project progresses and as new tools and technologies emerge.