SCTO Computerized Systems Validation Policy for R

Document development, review and version history

Development and Review

Name	Date
Michael Coslovsky,¹ Christine Otieno,² Alan Haynes³	2025-01-01

Version History

Version	Date	Author	Summary of Changes
1.0	2025-01-01	Michael Coslovsky, Christine Otieno, Alan Haynes	Initial draft

1 Executive summary

The typical guiding line of computerised system validation consists, in simple terms, of the following steps:

Define what you need
Plan how you confirm that you have what you need
Test that you have what you need
Document all steps from planning to test results.

Based on this concept, this policy provides a framework for member organizations of Swiss-Clinical Trial Organization (SCTO) to validate the language and environment “R”. In Section 7 of this policy we list and explain the documents that constitute a “validation documentation set”, as expected by regulatory entities examining or auditing computerised systems. We outline which of these documents are required and provide justification for those that are not required and, whenever possible, provide examples of relevant steps. Local adaptations to the principles outlined in this policy are required.

As explained in Section 7, using R under a validated environment touches on four different levels: infrastructure, R system level (primarily baseline installation), R-add on packages and the product generated using R. We provide guidance to the different levels. Risk assessment and management underlies all the decisions to the specifics required

The level of R add-on packages (Section 7.3) is the most challenging level, and potentially requires the most resources. To increase efficiency, we suggest here a unified process to first, assess an add-on package’s baseline risk, and second, to perform testing of functions that are part of R add-on packages. Together with the policy, we construct an online package inventory collecting these assessments and tests and making them available to all SCTO network members. As suggested in Section 8, the decision whether testing a function of an R add-on package is needed depends on the combined assessment of the add-on package’s baseline risk and of the risk associated with the specific product to be generated.

2 Introduction

This policy provides the framework for validating R and R add-on packages in use by Swiss Clinical Trial Organization (SCTO) and its SCTO network members⁴ for the purpose of creating R products in the context of clinical research. Validation may be executed on SCTO or CTU level, where the SCTO provides

recommendations for R base installation validation,
processes for R add-on package validation and
a platform to serve as an inventory for documentation of all R add-on package validation performed by the SCTO and its network members according to this policy.

Validation activities in general ensure that computerised systems can be relied upon to consistently produce a product that meets predetermined specifications of accuracy. The formal process of validation results in a validation documentation set⁵.

The validation of a computerised system consists of the following elements:

Infrastructure
Hardware
Software
Processes
Training

The validation activities focus on verifying that the software consistently fulfills the specified requirements. Processes must be in place to ensure that the infrastructure and hardware required for the functionality of the software are qualified and changes controlled. Additionally, all users and (if applicable) support personnel must be trained to use the software and follow related business processes.

According to international standards and guidelines listed below, any GxP⁶ critical system supporting activities such as statistical analysis (e.g., in the context of clinical studies) should be validated:

This SCTO policy is based on the ICH E6 guideline for Good Clinical Practice and the EMA guideline on computerised systems and electronic data in clinical trials. FDA regulations and guidelines (US) and Eudralex 4 (EU) represent additional best practices for validation of computerised systems in the industry and may serve as a guideline for academical purposes.

This policy is based on a business processes risk assessment provided by the SCTO statistics’ platform.

3 Background

R is a ‘language and environment for statistical computing and graphics’ (source: r-project.org). It is an official part of the Free Software Foundation’s GNU⁷ project and is released under the Free Software Foundation’s GNU Public License . Clinical Trial Units (CTU; see definition below) use R primarily, but not exclusively, in the framework of clinical research for tasks including sample size estimation and statistical design consideration, as well as the preparation and analysis of data from supported studies. Internal use for other reporting, visualisation and business management might also take place, according to individual CTU needs, but is not the focus of this policy.

Installation and use of R is based on several levels and sources:

The “official R distribution” (referred to as ‘R base installation’ in this document), which includes the R base and recommended R packages (as defined in the R: Regulatory compliance and validation issues – a guidance document, 2021) and is available via the Central R Archive Network (CRAN).
Distribution of contributed R add-on packages provided by the community and available via CRAN.
Distribution of R add-on packages provided by the community via other sources than CRAN (e.g., Bioconductor, Github, and various websites).

Additionally, R add-on packages may be developed in-house for internal use only and not distributed via any of the channels mentioned above. If such R add-on packages, or any of their functions, are used in the GxP context (e.g., clinical research), they must also be validated. This validation should, whenever feasible, follow the process outlined in this document and associated SOPs even if the R add-on package is not shared with the SCTO community.

4 Glossary of Terms & Abbreviations

CTU: Clinicial Trial Unit.

For the purposes of this document, all SCTO network members.

CRAN: Comprehensive R Archive Network
EMA: European Medicine Agency
FDA: US Food and Drug Administration
GxP: Good “x” Practice where “x” stands for the relevant area, e.g,: C = Clinical, M = Manufacturing, L = Laboratory
HLRA: High-Level Risk Assessment
HRA: (Swiss) Human Research Act
Intended for use package: "Intended for use” packages are add-on packages that comprise the functions required for a specific R product and are typically loaded directly by the user during an R session.

In this policy, when discussing the validation of an R add on package we refer to “intended for use” packages.

R Product

Any deliverable produced using R via the processes below. Typical statistical R products include sample size estimation, statistical analysis plan, an analysis report etc.

QMS

Quality Management System

R add-on package

Provided by members of the R community
available via CRAN and other distribution repositories
excludes Base R and R recommended packages

R Base Installation

Provided as part of the official R distribution released by the R Foundation
available via CRAN
includes Base R and R recommended packages

R recommended packages

A collection of packages, developed and validated by members of the R Development Core Team, and listed as ‘recommended’ in the document R: Regulatory Compliance and Validation Issues - A Guidance Document for the Use of R in Regulated Clinical Trial Environments.

SCTO

Swiss Clinical Trial Organisation

SOP

Standard Operating Procedure

Traceability

Refers to the ability to trace the development and validation processes a product underwent to understand its development history and make sure it fits its intended purpose. In detail, include the fulfillment of tests showing adherence to requirements, tracking software and package version and changes in product, the ability to reproduce results and documentation of protocols, scripts and results.

Validation documentation set

A set of documents that is prepared to plan and to provide evidence for the validation process.

5 General R Validation Approach

The SCTO follows a risk-based approach to the validation of R. For the validation and assessment of risks, we differentiate between the following levels, which build on each other:

IT Infrastructure level (IT infrastructure qualification according to local CTU processes⁸, see Section 7.1):
- A qualified IT infrastructure provides the basis for the validation of the R base installation and any R packages running on that installation.
R System level (R base installation, incl. recommended R packages, according to local CTU processes, recommendations and examples provided by SCTO, see Section 7.2):
- R base installation validation ensures that the basic installation of R (including recommended R packages) has been performed correctly and, thus, it provides a reliable framework on which additional R add-on packages can be installed and functions from those R add-on packages be executed.
R Package level (R add-on packages/ functions used within those packages, according to SCTO processes, see Section 7.3):
- R add-on package/ function validation ensures that all functions within an R add-on package required for a specific R product are sufficiently tested in relation to the R add-on package and R product risk (see SCTO R add-on Package Risk Assessment SOP for details). This takes place by evaluating the base risk associated with an R add-on package, and, upon necessity, performing specific function tests to ensure that functions (within an R add-on package) required for a specific R product produce consistent and reliable results from a statistical perspective (see SCTO R add-on Package Function Testing SOP for details).
R Product level (“intended use”/ outcome, according to by local CTU processes, see Section 7.4)

Important note: when creating an R product under a validated environment, the creator must ensure that

R is running on a qualified IT infrastructure,
using a validated base installation version, and
with proper documentation

to ensure consistency and reliability of results at all levels mentioned above.

6 Scope of this SCTO Validation Policy

6.1 In Scope

Recommendations for R base installation validation and periodic review of the R base installation validation documentation set (Section 7.2).

Note: The final validation documentation set and periodic review have to be defined and prepared by each CTU according to their local processes.

Processes for R add-on packages validation ("intended for use”) , incl. R add-on package risk assessment & functional testing as well as periodic review (Section 7.3).

Note: Processes for R add-on package validation described in this document are to be considered minimum requirements as shared by the SCTO platform. Any CTU providing input to the SCTO’s R add-on package validation repository has to follow at least those processes. Individual organizations (CTUs) can define stricter processes.

Recommendations for user training (Section 9).

6.2 Out of Scope

Standards for IT infrastructure (*to be defined on local level by the CTUs, see Section 7.1)
R add-on package management to ensure traceability and reproducibility on R product level (to be defined on local level by the CTUs, see Section 7.3)
Risk assessment and management of R products (to be defined on local level by the CTUs, see Section 7.4: R Products are always CTU project specific (e.g., a CTU’s clinical trial project) – a high-risk R product may result in the need for additional validation activities for an R add-on package, even if that R add-on package/required function is available as “validated” on the SCTO platform.
Internal, organization specific, processes of CTUs.

7 Detailed R Validation Approach

The following sections provide details on the required validation activities and documentation on all levels:

7.1 IT Infrastructure Level: IT Infrastructure Qualification
7.2 R System Level: R Base Installation Validation
7.3 R Package Level: R Add-on Package Validation
7.4 R Product Level: R Product Validation

Table 1 summarizes the required documents and activities for Section 7.2 and Section 7.3.

Table 1: R Base Installation Validation and R add-on Package Validation: Summary of Required Documents & Activities

Document/activity	Process
Document/activity	R Base installation (local CTU process)	R add-on Package validation
HLRA	required - SCTO recommendation	required
Vendor assessment	required - SCTO recommendation	not required
Validation plan + Test plan	required	not required
(User) requirements specification	required	not required
(Functional) risk assessment	required - SCTO recommendation	required
Software installation plan/ instruction	not required	not required
Test protocol ("User Acceptance Tests") / Test script	required	required
Installation Verification document (TEST)	required	not required
Executed test protocols ("User Acceptance Tests") / test scripts	required	required
Traceability Matrix	required	required
Validation & Test Report	required	required
Software Installation Verification document (PROD)	required	not required

7.1 IT Infrastructure Level: IT Infrastructure Qualification

The IT infrastructure qualification is managed at a local, institutional level, by the CTU or its organizational entity according to local IT processes.

For the reliable use of R and statistical traceability it is important to note which IT infrastructure (specifically which operating system software and version) R is installed on and to ensure that the infrastructure on which R is running is qualified.

7.2 R System Level: R Base Installation Validation

We recommend, that every CTU defines:

The minimal (validated) software version of the R Base Installation to be used at the CTU and a process of handling version changes.
The R base installation validation documentation set, providing evidence for the R base installation validation activities (see Table 3for recommendations).

Table 2 lists all documentation that is typically prepared during a computerised system validation process according to global standards and guidelines. It additionally provides recommendations how this documentation may be covered when preparing an R base installation validation documentation set. The exact content of the R base installation validation documentation set (i.e., the required documentation) at a CTU should follow the local processes and all documentation must be prepared and approved by the CTU. For some of the documents listed below, the SCTO provides examples (as indicated under “Detailed Description”).

Table 2: R Base Installation Validation: Detailed Description of Required Documents and Activities

Document or activity R Base installation	*Detailed Description:*	*Requirements*
	Purpose of the document/ activity and how this aspect can be covered for R (suggested R specific document/activity) Link to example document	If the document/ activity is NOT required for R: a rationale, why this document/activity can be omitted Recommended frequency of update
R Base installation High-Level Risk Assessment (HLRA; also referred to as System Risk Assessment)	Purpose: The purpose of such a document is to assess and document the risk of a system on a high level. It typically specifies at least the software category and the intended use of the software, including if it is used for GxP processes. The outcome of the HLRA (GxP relevance assessment) defines, if a system needs to be validated or not. Example: Example	Required for R base installation validation: Yes Frequency of update: Created before/ with the first R base installation validation. Updated, if any of the assessed aspects change. Reviewed during periodic review cycles. The review process should be documented even if no updates are required.
R Base installation Vendor Assessment	Purpose: The purpose of such a document is to the assess reliability of a system vendor. This may be done based on an audit or review of relevant vendor documentation. *R base validation specific approach: Since R is an open-source software managed by a consortium, a traditional vendor assessment/ audit approach is not feasible. A critical review of “R: Regulatory Compliance and Validation Issues - A Guidance Document for the Use of R in Regulated Clinical Trial Environments” may suffice as vendor assessment and should be documented together with any gaps identified that may require attention during the validation of R and its R add-on packages. Example:* Example	Required for R base installation validation: Yes Frequency of update: Created before/ with the first R base installation validation. Updated, if any of the assessed aspects change, specifically, with a publication of a new article version of the article mentioned above, or in case the R consortium replaces the existing document. Reviewed during periodic review cycles and review documented, if no updates are required.
R Base installation Validation Plan & Test Plan	Purpose: The purpose of such a document is to define the details of the planned validation process, including required documentation and acceptance criteria for productive use (Validation Plan). The Test Plan may be included in the Validation Plan or prepared as a separate document. The purpose of a Test Plan is to define the testing strategy and testing process and may contain details on the tests to be executed (if created for every new version validated). The exact content of such a document depends on the CTU’s local processes. Example: Example	Required for R base installation validation: Yes Frequency of update: Based on each CTU’s validation approach, either: Created with the first R base installation validation and updated only, if any aspects described in the plan change (in this case the plan would NOT contain any details about tests to be executed or specific document updates required for a newly released version. Instead, it would contain generic rules for test execution and document updates). If this approach is followed: Release-specific validation activities should be documented elsewhere (e.g., in a Change Plan) The document is only updated, if the validation and/or testing strategy changes. The document is reviewed during periodic review cycles and review documented, if no updates are required. Created or at least updated to a new document version with every validation of a new R base installation version (in this case the plan would contain all relevant details to plan and execute the release-specific validation). If this approach is followed An additional release-specific change plan may not be required. A new Validation & Test Plan or at least a new document version is created with every new R base installation version validation. No periodic review required.
R Base installation (User) Requirements Specification	Purpose: The purpose of such a document is to specify the user requirements and intended use for the R base installation. It may also contain any compliance (e.g., data integrity), regulatory (e.g., personal data protection) and safety requirements (e.g., controlled access). Example: Example	Required for R base installation validation: Yes Frequency of update: Created with the first R base installation validation. Updated, if any new requirements need to be added to cover the full scope of the current intended use OR if any of the existing requirements need to be updated or removed. Reviewed during periodic review cycles and review documented, if no updates are required.
R Base installation (Functional) Risk Assessment	Purpose: The purpose of such a document is to document the risk of system functionality based on the user requirements and/ or processes related to these requirements. Example: You may use the SCTO Statistics Business processes risk assessment document as a basis for this document	Required for R base installation validation: Yes Frequency of update: Created with the first R base installation validation. Updated, if any new functions or processes are added to the scope of the intended use of R. If new requirements are added to the User Requirements Specification, this is a good indicator to also review the Functional Risk Assessment document. Reviewed during periodic review cycles and review documented, if no updates are required.
R Base installation Software Installation Plan/Instruction	Purpose: The purpose of such a document would be to provide a process to follow during the installation of a software. It is typically provided either by the vendor or prepared by local IT departments. *R base validation specific approach:* It should be sufficient to follow the instructions on CRAN when downloading and installing the new R base installation version. Important is that the installation of the R base installation and version management follow the CTU’s local processes for validated systems and that all versions installed are documented, including the date/time of installation (see Installation Verification Document below).	Required for R base installation validation: No (see above for how this is covered) Frequency of update: N/A
R Base installation Test protocol (“User Acceptance Tests”)	Purpose: The purpose of such a document is to verify the functions of the R base installation supporting the user requirements. *R base validation specific approach: Test protocols provided by R may be used as-is or adapted/ extended as required. The scope of testing must be defined according to the CTU’s local processes and defined user requirements. Example:* An example will be available soon	Required for R base installation validation: Yes Frequency of update: Created with the first R base installation validation. Updated, if any new functions or processes are added to the scope of the intended use of R. If new requirements are added to the User Requirements Specification, this is a good indicator to also review the existing test protocols. Reviewed during periodic review cycles and review documented, if no updates are required.
R Base installation Software Installation Verification document (TEST)	Purpose: The purpose of such a document is to provide evidence of the installation of a specific system version on an environment/ machine that is used for testing purposes before the new system version is released to production. *R base validation specific approach: In case of R this may be a separate document or an entry in an R repository or other “version tracker” (depending on your local IT processes). The following information should be captured: R base installation version installed Date and time of installation Person performing the installation (may be someone from your local IT IT department) Outcome of installation: successful, successful with deviations, not successful If applicable: Any actions taken in addition to the regular “download and install from CRAN” process (e.g., uninstalling an older version) Note:* depending on your local processes it may not be feasible or even possible to do a preliminary “test installation” and perform tests before productive use of the new R base installation version (e.g., if R is distributed remotely by your local IT department). In this case, we recommend you still execute the tests after the installation and document the test execution. Should any issues be detected during testing, please inform your local IT department immediately. Ideally you implement a process that would ensure nobody is using the new R version for productive use before the tests are completed successfully (i.e., without major issues) and the validation & test report is approved (see “Validation & Test Report” below). Example: see Example for Validation & Test Report available	Required for R base installation validation: Yes Frequency of update: To be created with every new R base installation version to be validated for productive use No periodic review required
R Base installation Executed test protocols (“User Acceptance Tests”)	Purpose: The purpose of such a document is to provide evidence for the execution of the pre-defined test protocols for each new R base installation version (i.e., follow the steps described in the test protocols and document the outcome according to the CTU’s local processes). Example: An example will be available soon	Required for R base installation validation: Yes Frequency of update: To be created with every validation of a new R base installation version. No periodic review required
R Base installation Traceability Matrix	Purpose: The purpose of such a document is to ensure and provide evidence that all user requirements are tested during the validation process. You may omit testing low risk requirements, if such an approach is described in the Validation & Test Plan and complies to your CTU’s processes. Traceability between requirements and tests may be achieved by creating a separate document (traceability matrix) or by linking requirements and tests within the requirements and tests themselves. It is important, that whatever means is used, it is feasible to proof that all requirements are verified with a test (or if not tested formally, a rationale why the test is not required, e.g., low risk requirements which can be considered “verified” based on the experience by the user community). Example: An example will be available soon	Required for R base installation validation: Yes Frequency of update: Created with the first R base installation validation. Updated, if the user requirements specification and or the test protocols are updated. Reviewed during periodic review cycles and review documented, if no updates are required.
R Base installation Validation & Test Report	Purpose: The purpose of such a document is to summarize the testing activities and results, including any deviations from the expected results document all completed validation activities, including any deviations from the original plan (see Validation & Test Plan). Deviations should be justified and assessed for criticality. document the acceptance for productive use (with or without restrictions) The exact content of this report depends on the CTU’s local processes. Separate documents may be created to summarize the testing activities and results (Test Report) and the overall executed validation against the plan, incl. a statement of acceptance of an R base installation version for productive use (Validation Report). Example: Example	Required for R base installation validation: Yes Frequency of update: To be created with every new R base installation version validated for productive use No periodic review required
R Base installation Software Installation Verification document (PROD)	Purpose: The purpose of such a document is to provide evidence of the installation of a specific system version on an environment/ machine that is used for productive purposes. *R base validation specific approach: In case of R, this may be a separate document or an entry in an R repository or other “version tracker” (depending on your local IT processes). The following information should be captured: R base installation version installed Date and time of installation Person performing the installation (may be someone from your local IT department) Outcome of installation: successful, successful with deviations, not successful If applicable: Any actions taken in addition to the regular “download and install from CRAN” process (e.g., uninstalling an older version) See also note* under “Installation Verification document (TEST)”: In case of R base installation, the installation may only be documented once and not separately on “TEST” and “PROD”. Important is, that any version changes of the R base installation are managed and documented according to your CTU’s local IT change management processes.	Required for R base installation validation: Yes Frequency of update: To be created with every new R base installation version validated for productive use No periodic review required

Purpose of the document/ activity and how this aspect can be covered for R (suggested R specific document/activity)
Link to example document

If the document/ activity is NOT required for R: a rationale, why this document/activity can be omitted
Recommended frequency of update

R Base installation

High-Level Risk Assessment (HLRA; also referred to as System Risk Assessment)

Purpose:

The purpose of such a document is to assess and document the risk of a system on a high level. It typically specifies at least the software category and the intended use of the software, including if it is used for GxP processes. The outcome of the HLRA (GxP relevance assessment) defines, if a system needs to be validated or not.

Example:

Example

Required for R base installation validation:

Frequency of update:

Created before/ with the first R base installation validation.
Updated, if any of the assessed aspects change.
Reviewed during periodic review cycles. The review process should be documented even if no updates are required.

R Base installation

Vendor Assessment

Purpose:

The purpose of such a document is to the assess reliability of a system vendor. This may be done based on an audit or review of relevant vendor documentation.

R base validation specific approach: Since R is an open-source software managed by a consortium, a traditional vendor assessment/ audit approach is not feasible. A critical review of “R: Regulatory Compliance and Validation Issues - A Guidance Document for the Use of R in Regulated Clinical Trial Environments” may suffice as vendor assessment and should be documented together with any gaps identified that may require attention during the validation of R and its R add-on packages.

Example:

Example

Required for R base installation validation:

Frequency of update:

Created before/ with the first R base installation validation.
Updated, if any of the assessed aspects change, specifically, with a publication of a new article version of the article mentioned above, or in case the R consortium replaces the existing document.
Reviewed during periodic review cycles and review documented, if no updates are required.

R Base installation

Validation Plan & Test Plan

Purpose:

The purpose of such a document is to define the details of the planned validation process, including required documentation and acceptance criteria for productive use (Validation Plan). The Test Plan may be included in the Validation Plan or prepared as a separate document. The purpose of a Test Plan is to define the testing strategy and testing process and may contain details on the tests to be executed (if created for every new version validated). The exact content of such a document depends on the CTU’s local processes.

Example:

Example

Required for R base installation validation:

Frequency of update:

Based on each CTU’s validation approach, either:
- Created with the first R base installation validation and updated only, if any aspects described in the plan change (in this case the plan would NOT contain any details about tests to be executed or specific document updates required for a newly released version. Instead, it would contain generic rules for test execution and document updates). If this approach is followed:
  - Release-specific validation activities should be documented elsewhere (e.g., in a Change Plan)
    - The document is only updated, if the validation and/or testing strategy changes.
    - The document is reviewed during periodic review cycles and review documented, if no updates are required.
- Created or at least updated to a new document version with every validation of a new R base installation version (in this case the plan would contain all relevant details to plan and execute the release-specific validation). If this approach is followed
  - An additional release-specific change plan may not be required.
  - A new Validation & Test Plan or at least a new document version is created with every new R base installation version validation.
  - No periodic review required.

R Base installation

(User) Requirements Specification

Purpose:

The purpose of such a document is to specify the user requirements and intended use for the R base installation. It may also contain any compliance (e.g., data integrity), regulatory (e.g., personal data protection) and safety requirements (e.g., controlled access).

Example:

Example

Required for R base installation validation:

Frequency of update:

Created with the first R base installation validation.
Updated, if any new requirements need to be added to cover the full scope of the current intended use OR if any of the existing requirements need to be updated or removed.
Reviewed during periodic review cycles and review documented, if no updates are required.

R Base installation

(Functional) Risk Assessment

Purpose:

The purpose of such a document is to document the risk of system functionality based on the user requirements and/ or processes related to these requirements.

Example:

You may use the SCTO Statistics Business processes risk assessment document as a basis for this document

Required for R base installation validation:

Frequency of update:

Created with the first R base installation validation.
Updated, if any new functions or processes are added to the scope of the intended use of R. If new requirements are added to the User Requirements Specification, this is a good indicator to also review the Functional Risk Assessment document.
Reviewed during periodic review cycles and review documented, if no updates are required.

R Base installation

Software Installation Plan/Instruction

Purpose:

The purpose of such a document would be to provide a process to follow during the installation of a software. It is typically provided either by the vendor or prepared by local IT departments.

R base validation specific approach: It should be sufficient to follow the instructions on CRAN when downloading and installing the new R base installation version.

Important is that the installation of the R base installation and version management follow the CTU’s local processes for validated systems and that all versions installed are documented, including the date/time of installation (see Installation Verification Document below).

Required for R base installation validation:

No (see above for how this is covered)

Frequency of update:

R Base installation

Test protocol (“User Acceptance Tests”)

Purpose:

The purpose of such a document is to verify the functions of the R base installation supporting the user requirements.

R base validation specific approach: Test protocols provided by R may be used as-is or adapted/ extended as required. The scope of testing must be defined according to the CTU’s local processes and defined user requirements.

Example:

An example will be available soon

Required for R base installation validation:

Frequency of update:

Created with the first R base installation validation.
Updated, if any new functions or processes are added to the scope of the intended use of R. If new requirements are added to the User Requirements Specification, this is a good indicator to also review the existing test protocols.
Reviewed during periodic review cycles and review documented, if no updates are required.

R Base installation

Software Installation Verification document (TEST)

Purpose:

The purpose of such a document is to provide evidence of the installation of a specific system version on an environment/ machine that is used for testing purposes before the new system version is released to production.

R base validation specific approach: In case of R this may be a separate document or an entry in an R repository or other “version tracker” (depending on your local IT processes). The following information should be captured:

R base installation version installed
Date and time of installation
Person performing the installation (may be someone from your local IT IT department)
Outcome of installation: successful, successful with deviations, not successful
If applicable: Any actions taken in addition to the regular “download and install from CRAN” process (e.g., uninstalling an older version)

Note: depending on your local processes it may not be feasible or even possible to do a preliminary “test installation” and perform tests before productive use of the new R base installation version (e.g., if R is distributed remotely by your local IT department). In this case, we recommend you still execute the tests after the installation and document the test execution. Should any issues be detected during testing, please inform your local IT department immediately. Ideally you implement a process that would ensure nobody is using the new R version for productive use before the tests are completed successfully (i.e., without major issues) and the validation & test report is approved (see “Validation & Test Report” below).

Example:

see Example for Validation & Test Report available

Required for R base installation validation:

Frequency of update:

To be created with every new R base installation version to be validated for productive use
No periodic review required

R Base installation

Executed test protocols (“User Acceptance Tests”)

Purpose:

The purpose of such a document is to provide evidence for the execution of the pre-defined test protocols for each new R base installation version (i.e., follow the steps described in the test protocols and document the outcome according to the CTU’s local processes).

Example:

An example will be available soon

Required for R base installation validation:

Frequency of update:

To be created with every validation of a new R base installation version.
No periodic review required

R Base installation

Traceability Matrix

Purpose:

The purpose of such a document is to ensure and provide evidence that all user requirements are tested during the validation process. You may omit testing low risk requirements, if such an approach is described in the Validation & Test Plan and complies to your CTU’s processes.

Traceability between requirements and tests may be achieved by creating a separate document (traceability matrix) or by linking requirements and tests within the requirements and tests themselves. It is important, that whatever means is used, it is feasible to proof that all requirements are verified with a test (or if not tested formally, a rationale why the test is not required, e.g., low risk requirements which can be considered “verified” based on the experience by the user community).

Example:

An example will be available soon

Required for R base installation validation:

Frequency of update:

Created with the first R base installation validation.
Updated, if the user requirements specification and or the test protocols are updated.
Reviewed during periodic review cycles and review documented, if no updates are required.

R Base installation

Validation & Test Report

Purpose:

The purpose of such a document is to

summarize the testing activities and results, including any deviations from the expected results
document all completed validation activities, including any deviations from the original plan (see Validation & Test Plan). Deviations should be justified and assessed for criticality.
document the acceptance for productive use (with or without restrictions)

The exact content of this report depends on the CTU’s local processes. Separate documents may be created to summarize the testing activities and results (Test Report) and the overall executed validation against the plan, incl. a statement of acceptance of an R base installation version for productive use (Validation Report).

Example:

Example

Required for R base installation validation:

Frequency of update:

To be created with every new R base installation version validated for productive use
No periodic review required

R Base installation

Software Installation Verification document (PROD)

Purpose:

The purpose of such a document is to provide evidence of the installation of a specific system version on an environment/ machine that is used for productive purposes.

R base validation specific approach: In case of R, this may be a separate document or an entry in an R repository or other “version tracker” (depending on your local IT processes). The following information should be captured:

R base installation version installed
Date and time of installation
Person performing the installation (may be someone from your local IT department)
Outcome of installation: successful, successful with deviations, not successful
If applicable: Any actions taken in addition to the regular “download and install from CRAN” process (e.g., uninstalling an older version)

See also note under “Installation Verification document (TEST)”: In case of R base installation, the installation may only be documented once and not separately on “TEST” and “PROD”. Important is, that any version changes of the R base installation are managed and documented according to your CTU’s local IT change management processes.

Required for R base installation validation:

Frequency of update:

To be created with every new R base installation version validated for productive use
No periodic review required

7.3 R Package Level: R Add-on Package Validation

The R add-on package validation documentation set provides evidence of the validation process of an “intended for use” R add-on package following this policy and associated SOPs. Table 3 lists all documentation that is typically prepared during a computerised system validation process. It additionally provides details on how this documentation is covered for an R add-on package validation according to this policy.

NOTE on dependencies: R add-on packages often come with ‘dependencies’, namely additional add-on packages required for the proper utilization of the specified R add-on package. Package dependency ‘trees’ can be very large and complex, raising the challenge of validation of the dependencies as well. Validating at a single step a specific R add-on package with all its dependencies is thus almost unfeasible. Hence, in this policy, when referring to the validation of an add-on package, the meaning is the “intended for use” R add-on package itself and its main functions, leaving dependencies unvalidated.

Table 3: R add-on Package Validation: Detailed Description of Required Documents and Activities

Document or activity R add-on package	*Detailed Description:*	*Requirements*
	Purpose of the document/ activity and how this aspect can be covered for R (suggested R specific document/ activity) Link to example/relevant SCTO document	If the document/ activity is NOT required for R: a rationale, why this document/ activity can be omitted Recommended frequency of update
R add-on Package High-Level Risk Assessment (HLRA; also referred to as System Risk Assessment)	Purpose: See Table 3: R Base Installation Validation: Detailed Description of Required Documents and Activities. *R add-on package specific approach: The R add-on package risk depends on factors described in the SCTO R add-on Package Risk Assessment SOP. Accordingly, the R add-on package high-level risk assessment should be documented according to the SCTO R Package Risk Assessment SOP on the SCTO designated platform. Relevant SCTO SOP:* SCTO R add-on Package Risk Assessment SOP	Required for R add-on package validation: Yes (see above for how this is covered) Frequency of update: See Section 10
R add-on Package Vendor Assessment	Purpose: See Table 3: R Base Installation Validation: Detailed Description of Required Documents and Activities . *R add-on package specific approach: Covered by the Vendor Assessment created for the R Base Installation and the SCTO R add-on Package Risk Assessment SOP. Relevant SCTO SOP:* SCTO R Validation Policy (this document) SCTO R add-on Package Risk Assessment SOP	Required for R add-on package validation: No (see above for how this is covered) Frequency of update: See Section 10
R add-on Package Validation & Test Plan	Purpose: The purpose of such a document would be to define the details of the planned validation process, including required documentation and acceptance criteria for productive use (Validation Plan). The Test Plan may be included in the Validation Plan or prepared as a separate document. The purpose of a Test Plan is to define the testing strategy and testing process and may contain details on the tests to be executed (if created for every new version validated). *R add-on package specific approach: The content typically contained in a Validation & Test Plan is determined, with respect to R add-on packages, in the various steps of this policy and the associated SOPs and its documentation is specified therein as follows: Determine the R product associated risk (local CTU SOP) Determine the R add-on package associated risk (based on SCTO R Package Risk Assessment SOP) Determine, if testing is required (see Table 5: Assessment of Combined Risk R add-on Package and R Product) Perform and document testing, if required (SCTO R add-on Package Function Testing SOP) Document compliance with this policy and above-mentioned SOPs (local CTU process, e.g., in metadata or in a document) Relevant SCTO Documentation:* SCTO R Validation Policy (this document) SCTO R add-on Package Risk Assessment SOP SCTO R add-on Package Function Testing SOP	Required for R add-on package validation: No (see above for how this is covered) Frequency of update: See Section 10
R add-on Package (User) Requirements Specification	Purpose: See Table 3: R Base Installation Validation: Detailed Description of Required Documents and Activities. *R add-on package specific approach: The requirements and intended use of an R add-on package and its functions can only be defined in relation to an R product and the required outcome (“intended use”) of a specific function for that R product. Therefore, the requirements can only be documented when planning the R product. Requirements of the product, which guide the selection of R add-on packages and functions therein, shall be documented in the Statistical Analysis Plan, a Statistical Protocolor other, similar documentation⁹. Relevant SCTO Documentation:* N/A (processes for planning R products and defining their intended use have to be defined on CTU level)	Required for R add-on package validation: No (see above for how this is covered) Frequency of update: Upon changes or deviations from the originally intended use/ requirements defined in the plan for an R product change.
R add-on Package Function Risk Assessment	Purpose: The purpose of such a document would be to specify the risks related to a specific function. *R add-on package specific approach: We take a package level approach - the assessed baseline risk of the R add-on package defines the risk of all functions included within. Relevant SCTO SOP:* - SCTO R add-on Package Risk Assessment SOP	Required for R add-on package validation: No (covered by the R add-on Package High-level Risk assessment) Frequency of update: NA
R add-on Package Software Installation Plan/Instruction	Purpose: See Table 3: R Base Installation Validation: Detailed Description of Required Documents and Activities. *R add-on package specific approach: The installation follows a standard procedure using R base installation functions. R add-on packages may be installed for functional testing* after the high-level R add-on package risk and the R product related requirements and functional risk are documented and the testing is planned, if the initial assessment shows that the required R add-on package is not yet sufficiently tested by one of the SCTO members providing input to the R add-on package validation repository. *R add-on packages may be installed for productive use/ used to create R products* after the functional tests are completed and no deviations were detected that would raise concerns against using the tested R add-on package functions for a given R product OR if the initial assessment shows that the required R add-on package was already sufficiently tested and the respective validation documentation set is available in the SCTO R add-on package validation repository. Relevant SCTO Documentation: N/A (standard R process, CTU specific IT operations SOPs may exist to cover local specifics of installation)	Required for R add-on package validation: No (see above for how this is covered) Frequency of update: N/A for standard R process According to CTU-specific periodic review cycle
R add-on Package Test scripts	Purpose: The purpose of such a document is to verify the functions of the R add-on package work as expected. *R add-on package specific approach: The process of writing and executing tests for the R add-on packages is covered in the SCTO R add-on Package Function Testing SOP. Relevant SCTO SOP:* SCTO R add-on Package Function Testing SOP	Required for R add-on package validation: Yes (see above for how this is covered) Frequency of update: See SCTO R add-on Package Function Testing SOP (check of appropriateness and sufficiency of an existing test script)
R add-on Package Executed Software installation verification (TEST)	Purpose: The purpose of such a document would be to provide evidence of the installation of a specific R add-on package (version) for testing purposes. *R add-on package specific approach: In the context of R add-on package validation it is not* feasible to create a separate document. The tested R add-on package (version) functions are documented when following the SCTO R add-on Package Function Testing SOP. Relevant SCTO SOP: SCTO R add-on Package Function Testing SOP	Required for R add-on package validation: No (see above for how this is covered) Frequency of update: NA
R add-on Package Executed Test Scripts	Purpose: The purpose of such a document is to provide evidence for the execution of the pre-defined test scripts. *R add-on package specific approach: Execute the pre-defined test scripts and document the outcome of each step as well as the overall outcome of the test script according to SCTO R add-on Package Function Testing SOP. Relevant SCTO SOP:* SCTO R add-on Package Function Testing SOP	Required for R add-on package validation: Yes (see above for how this is covered) Frequency of update: N/A (test executions are never updated. Test scripts may be re-executed, if required)
R add-on Package Traceability Matrix	Purpose: The purpose of such a document is to ensure and provide evidence that all functions in scope are tested during the validation process. You may omit testing low risk functions. *R add-on package specific approach: Covered by SCTO R add-on Package Risk Assessment SOP and SCTO R add-on Package Function Test SOP. Relevant SCTO SOP:* SCTO R add-on Package Risk Assessment SOP SCTO R add-on Package Function Testing SOP	Required for R add-on package validation: Yes (Documentation of function testing according to SCTO R add-on Package Function Test SOP includes info on the tested function) Frequency of update: Depends on the update and execution of tests for specific functions.
R add-on Package Validation & Test Report	Purpose: The purpose of such a document is to summarise all executed validation activities and their outcome, including testing and any defects found and/or any deviations from the original plan with a rationale. It documents the acceptance for productive use (with or without restrictions) of specific functions from an R add-on package (version) (see SCTO R add-on Package Function Testing SOP). Relevant SCTO SOP: SCTO R add-on Package Function Testing SOP	Required for R add-on package validation: Yes Frequency of update: See process described in SCTO R add-on Package Function Testing SOP

Purpose of the document/ activity and how this aspect can be covered for R (suggested R specific document/ activity)
Link to example/relevant SCTO document

If the document/ activity is NOT required for R: a rationale, why this document/ activity can be omitted
Recommended frequency of update

R add-on Package

High-Level Risk Assessment (HLRA; also referred to as System Risk Assessment)

Purpose:

See Table 3: R Base Installation Validation: Detailed Description of Required Documents and Activities.

R add-on package specific approach: The R add-on package risk depends on factors described in the SCTO R add-on Package Risk Assessment SOP. Accordingly, the R add-on package high-level risk assessment should be documented according to the SCTO R Package Risk Assessment SOP on the SCTO designated platform.

Relevant SCTO SOP:

SCTO R add-on Package Risk Assessment SOP

Required for R add-on package validation:

Yes (see above for how this is covered)

Frequency of update:

See Section 10

R add-on Package

Vendor Assessment

Purpose:

See Table 3: R Base Installation Validation: Detailed Description of Required Documents and Activities .

R add-on package specific approach: Covered by the Vendor Assessment created for the R Base Installation and the SCTO R add-on Package Risk Assessment SOP.

Relevant SCTO SOP:

SCTO R Validation Policy (this document)
SCTO R add-on Package Risk Assessment SOP

Required for R add-on package validation:

No (see above for how this is covered)

Frequency of update:

See Section 10

R add-on Package

Validation & Test Plan

Purpose:

The purpose of such a document would be to define the details of the planned validation process, including required documentation and acceptance criteria for productive use (Validation Plan). The Test Plan may be included in the Validation Plan or prepared as a separate document. The purpose of a Test Plan is to define the testing strategy and testing process and may contain details on the tests to be executed (if created for every new version validated).

R add-on package specific approach: The content typically contained in a Validation & Test Plan is determined, with respect to R add-on packages, in the various steps of this policy and the associated SOPs and its documentation is specified therein as follows:

Determine the R product associated risk (local CTU SOP)
Determine the R add-on package associated risk (based on SCTO R Package Risk Assessment SOP)
Determine, if testing is required (see Table 5: Assessment of Combined Risk R add-on Package and R Product)
Perform and document testing, if required (SCTO R add-on Package Function Testing SOP)
Document compliance with this policy and above-mentioned SOPs (local CTU process, e.g., in metadata or in a document)

Relevant SCTO Documentation:

SCTO R Validation Policy (this document)
SCTO R add-on Package Risk Assessment SOP
SCTO R add-on Package Function Testing SOP

Required for R add-on package validation:

No (see above for how this is covered)

Frequency of update:

See Section 10

R add-on Package

(User) Requirements Specification

Purpose:

See Table 3: R Base Installation Validation: Detailed Description of Required Documents and Activities.

R add-on package specific approach: The requirements and intended use of an R add-on package and its functions can only be defined in relation to an R product and the required outcome (“intended use”) of a specific function for that R product. Therefore, the requirements can only be documented when planning the R product.

Requirements of the product, which guide the selection of R add-on packages and functions therein, shall be documented in the Statistical Analysis Plan, a Statistical Protocolor other, similar documentation⁹.

Relevant SCTO Documentation:

N/A (processes for planning R products and defining their intended use have to be defined on CTU level)

Required for R add-on package validation:

No (see above for how this is covered)

Frequency of update:

Upon changes or deviations from the originally intended use/ requirements defined in the plan for an R product change.

R add-on Package

Function Risk Assessment

Purpose:

The purpose of such a document would be to specify the risks related to a specific function.

R add-on package specific approach: We take a package level approach - the assessed baseline risk of the R add-on package defines the risk of all functions included within.

Relevant SCTO SOP: - SCTO R add-on Package Risk Assessment SOP

Required for R add-on package validation:

No (covered by the R add-on Package High-level Risk assessment)

Frequency of update:

R add-on Package

Software Installation Plan/Instruction

Purpose:

See Table 3: R Base Installation Validation: Detailed Description of Required Documents and Activities.

R add-on package specific approach: The installation follows a standard procedure using R base installation functions.

R add-on packages may be installed for functional testing after the high-level R add-on package risk and the R product related requirements and functional risk are documented and the testing is planned, if the initial assessment shows that the required R add-on package is not yet sufficiently tested by one of the SCTO members providing input to the R add-on package validation repository.
R add-on packages may be installed for productive use/ used to create R products after the functional tests are completed and no deviations were detected that would raise concerns against using the tested R add-on package functions for a given R product OR if the initial assessment shows that the required R add-on package was already sufficiently tested and the respective validation documentation set is available in the SCTO R add-on package validation repository.

Relevant SCTO Documentation:

N/A (standard R process, CTU specific IT operations SOPs may exist to cover local specifics of installation)

Required for R add-on package validation:

No (see above for how this is covered)

Frequency of update:

N/A for standard R process
According to CTU-specific periodic review cycle

R add-on Package

Test scripts

Purpose:

The purpose of such a document is to verify the functions of the R add-on package work as expected.

R add-on package specific approach: The process of writing and executing tests for the R add-on packages is covered in the SCTO R add-on Package Function Testing SOP.

Relevant SCTO SOP:

SCTO R add-on Package Function Testing SOP

Required for R add-on package validation:

Yes (see above for how this is covered)

Frequency of update:

See SCTO R add-on Package Function Testing SOP (check of appropriateness and sufficiency of an existing test script)

R add-on Package

Executed Software installation verification (TEST)

Purpose:

The purpose of such a document would be to provide evidence of the installation of a specific R add-on package (version) for testing purposes.

R add-on package specific approach: In the context of R add-on package validation it is not feasible to create a separate document. The tested R add-on package (version) functions are documented when following the SCTO R add-on Package Function Testing SOP.

Relevant SCTO SOP:

SCTO R add-on Package Function Testing SOP

Required for R add-on package validation:

No (see above for how this is covered)

Frequency of update:

R add-on Package

Executed Test Scripts

Purpose:

The purpose of such a document is to provide evidence for the execution of the pre-defined test scripts.

R add-on package specific approach: Execute the pre-defined test scripts and document the outcome of each step as well as the overall outcome of the test script according to SCTO R add-on Package Function Testing SOP.

Relevant SCTO SOP:

SCTO R add-on Package Function Testing SOP

Required for R add-on package validation:

Yes (see above for how this is covered)

Frequency of update:

N/A (test executions are never updated. Test scripts may be re-executed, if required)

R add-on Package

Traceability Matrix

Purpose:

The purpose of such a document is to ensure and provide evidence that all functions in scope are tested during the validation process. You may omit testing low risk functions.

R add-on package specific approach: Covered by SCTO R add-on Package Risk Assessment SOP and SCTO R add-on Package Function Test SOP.

Relevant SCTO SOP:

Required for R add-on package validation:

Yes (Documentation of function testing according to SCTO R add-on Package Function Test SOP includes info on the tested function)

Frequency of update:

Depends on the update and execution of tests for specific functions.

R add-on Package

Validation & Test Report

Purpose:

The purpose of such a document is to summarise all executed validation activities and their outcome, including testing and any defects found and/or any deviations from the original plan with a rationale. It documents the acceptance for productive use (with or without restrictions) of specific functions from an R add-on package (version) (see SCTO R add-on Package Function Testing SOP).

Relevant SCTO SOP:

SCTO R add-on Package Function Testing SOP

Required for R add-on package validation:

Frequency of update:

See process described in SCTO R add-on Package Function Testing SOP

7.4 R Product Level: R Product Validation

To be covered by local SOPs for statistical analysis execution at the CTUs. The R product validation should cover at minimum:

Assessment of risk associated with the R product
Assess the tools used within the R product
Documentation for reproducibility and traceability

8 Risk Management

Given the different validation levels, the risk also needs to be assessed, documented, and managed on all those levels, specifically on:

IT infrastructure level: The risk associated with IT infrastructure components can generally assumed to be low.
R System level (R base installation and R recommended packages, *recommendations provided in this policy, documentation according to local CTU processes, see Section 7.2)
R Package level (R add-on packages, see SCTO R add-on Package Risk Assessment SOP)
R Product level (according to local CTU processes, see Section 7.4)

The actions required from a user, with respect to a specific product, depend on the combined risk of the R add-on package (baseline risk) and the product risk. More intensive, low-level, actions, such as specific function testing, are required for higher risk combinations. The requirements for testing a function of an add-on R package are defined in Table 5.

		R add-on Package Risk (Baseline)
		Low	Medium	High
R Product risk	Low	No function testing required	No function testing required	No function testing required
	Medium	No function testing required	No function testing required	Function testing required
	High	No function testing required	Function testing required	Function testing required

9 User Training

All users performing statistical analysis and/or developing R add-on packages and creating R products need to be qualified and trained according to their CTU’s guidelines.

All users (potentially) involved in validation activities (e.g., risk assessments, testing) must additionally be trained on this policy and the associated SOPs.

10 Periodic Review

Every 3 years the validation status of the R add-on packages in the SCTO validation inventory should be reviewed, if R add-on packages and/or functions were not updated and re-tested within that period. The review needs to be documented, even if no updates are required.

The periodic review of the R base installation validation documentation set has to be defined and managed according to the CTU’s processes. We recommend including a review of the currently defined minimum R base installation version in the periodic review process.

11 Associated Documents

R add-on Package Risk Assessment SOP: This document describes the process and metrics defining the risk of R add-on packages.
R add-on Package Function Testing SOP: This document describes the standard process for testing specific functions within an R add-on package.

Note: Documents to be prepared at CTU level are not listed here. When possible, examples for selected documents are listed in relevant sections.

12 References

Table 4: References

Reference Title	Description
Guideline on computerised systems and electronic data in clinical trials	European Medicines Agency: Guideline on computerised systems and electronic data in clinical trials (2023)
ICH E6: Guideline for Good Clinical Practice	International Council for Harmonisation, E6 Good Clinical Practice (1997) and Integrated Addendum to Good Clinical Practice (GCP) (2016)
The Good Automated Manufacturing Practice (GAMP) Guide 5: A Risk Based Approach to Compliant GxP Computerized Systems (ISPE/GAMP, 2 008)	Guideline for computerised systems validation in regulated environments issued by the International Society for Pharmaceutical Engineering (ISPE). Note that the content of GAMP 5 is not available online.
FDA 21 CFR part 11: Electronic Records; Electronic Signatu res	FDA regulation: Title 21—Food and Drugs, Chapter I - Food and Drug Administration, Department of Health and Human Services, Subchapter A - General, Part 11 — Electronic Records; Electronic Signatures
General Principles of Software Validation; Final Guidance for Industry and FDA Staff (2002)	FDA guidance paper for computerised system validation.
EudraLex Volume 4: Good Manufacturing Practice (GMP) guidelines, Annex 11: Computerised Syst ems	European standard for comupterised systems validation.
R: Regulatory Compliance and Validation Issues - A Guidance Document for the Use of R in Regulated Clinical Trial Environm ents	R Foundation for Statistical Computing guidance document for the validation of R for the use in regulated environments.

Footnotes

Head data-analysis, Department Clinical Research, University of Basel↩︎
IT Quality Manager, Department Clinical Research, University of Basel↩︎
Senior Statistician, Department of Clinical Research (DCR), University of Bern↩︎
for purposes of this document referred to as CTUs = Clinical Trial Units↩︎
A validation documentation set is prepared to provide evidence of the validation process.↩︎
GxP processes refer to Good Practice standards, including Good Clinical Practice (GCP), Good Manufacturing Practices (GMP), Good Laboratory Practice (GLP) etc. GxP critical computerised systems are all systems that manage GxP data (e.g., clinical study data) and therefore support and/or provide input for GxP processes.↩︎
The GNU Project is a free software, mass collaboration project announced by Richard Stallman on September 27, 1983. Its goal is to give computer users freedom and control in their use of their computers and computing devices by collaboratively developing and publishing software that gives everyone the rights to freely run the software, copy and distribute it, study it, and modify it. GNU software grants these rights in its license (https://en.wikipedia.org/wiki/GNU_Project).↩︎
For example, standards set by the local IT department.↩︎
Examples of such other documents could be the defined research question when planning a sample-size/power estimation, description of data-base/shiny-app user requirements, etc.↩︎