TMF completeness findings are the most commonly reported inspection findings. For example, in them most recently published MHRA GCP inspection report , there were twice as many findings in this category than in the next most common category (ancillary systems).

Completeness is an ephemeral metric – it changes every day for an active study, as documents are added and finalized and milestone and other due dates change. This contributes to making completeness the most challenging aspect of TMF management. Is that because completeness in unknown, or because it is unknowable? In this post, we will examine the factors that contribute to an accurate view of completeness, and provide some practical advice that will help you evolve towards a more accurate assessment of completeness.

Want to learn more? Check out our webinar series: eTMF for Risk Management and Regulatory Compliance

• Episode 1: Guidance Review
• Episode 2: Completeness (currently viewing)
• Episode 3: Timeliness
• Episode 4: Quality
• Episode 5: Inspections
• Episode 6: Sponsor Access, Wrap-Up

The Impact of Due Dates

To understand a TMF’s completeness, it’s necessary to have a good understanding of when documents are actually needed. On day 1 of a trial, the Clinical Study Report is not “missing” or “overdue” – it’s not expected yet. For an ongoing trial, completeness needs to be assed based on what is actually expected in the TMF at any given point in time.
Modern eTMFs determine quality based on what documents are due at any point in a trial (not just at the end). But how do you determine when documents are “due”?

Due dates can originate from many sources (not just milestones), as shown in the figure below. To know what is due on any given date, an accurate due date must be assigned, and kept up to date as the trial evolves. Some eTMFs provide extensive features for managing due dates, but some provide only basic functionality or none at all.

Sophistication of Planning Processes

Most eTMFs provide a mechanism for planning the set of documents that are needed for a TMF. These many take the form of placeholders, expected document lists, counters, etc. This is equivalent to the trial manifest or index spreadsheet kept for the TMF before it became electronic.

The detail and accuracy of this planning process has a major impact on completeness. To understand this better, let’s look at some examples.

Example 1: Kickoff Meeting

The TMF Reference Model includes the following artifact and sub-artifacts for the kickoff meeting. Let’s assume you need one of each sub-artifact to be complete.

The eTMF, based on its functionality and configuration, might plan any of the following:

1. Automatically plan 1 placeholder for each sub-artifact, named with the sub-artifact. Most accurate list of expected documents.
2. Automatically plan 4 placeholders named “Kick Off Meeting Materials.” Gives an idea of what is needed, but not enough information to allow users to have a clear understanding of what is missing.
3. Automatically plan a single placeholder for “Kick Off Meeting Materials.” Gives a misleading picture of what is needed and skews completeness.
4. Nothing, they have to be planned manually. Obviously a time-consuming and error-prone approach.

Example 2: New Lab Added Mid-Trial

Suppose there is a need to add a new laboratory at some time after the trial has begun.  Of course, documents will be needed for the new lab.  In this case, the TMF Reference Model has an entire set of artifacts, each with various sub-artifacts:

Again, the eTMF might update the required documents in a number of ways:

1. Automatically plan 1 placeholder for each required artifact and sub-artifact, named with the artifact/sub-artifact and lab name
2. Automatically plan 1 placeholder for each required artifact and sub-artifact, named with the artifact/sub-artifact
3. Plan 1 placeholder for each artifact
4. Nothing, they have to be planned manually

It’s easy to see how 3 and 4 result in a significant undercount of required documents and resulting inaccuracy in assessing completeness.  Outcome 2 creates planned documents but does not provide clarity on which lab they are needed for.

Unplanned Documents

Many documents can’t really be planned in advance, as shown in the figure below:

Because these documents won’t be in any set of expected documents or placeholders, sound TMF management processes are usually the only way to assess their completeness.  Again, failure to manage this aspect of TMF will lead to an inflated assessment of completeness that will often be discovered by a Health Authority during an inspection.

Ancillary Systems

Of course, all of the documents and records that actually comprise the TMF are not necessarily stored in the main eTMF.  These records can be found in a variety of repositories as shown below.

When records aren’t stored in the main TMF, responsibilities for ensuring completeness in their systems of record must be established.  Health authorities will almost certainly require access to some of them during an inspection, so procedures for ensuring and monitoring completeness are needed.

Conclusion

This post has highlighted many of the reasons why it’s difficult and complicated to track TMF completeness.  But this doesn’t mean that you can’t take proactive steps to improve completeness.  Here are some recommendations.

1. TMF will never be perfect – focus on Critical to Quality factors, risk, and process improvement.
2. Data is Power. Having a complete, thoughtfully defined set of metrics will allow you to learn about and improve your processes.
3. Review your TMF tool for planning weaknesses.  If you understand where completeness predictions are poor (and why), you may be able to re-examine the tool’s capabilities to improve accuracy.  If that’s not possible, you can augment with process.
4. Re-examine your processes around capturing unplanned documents in a timely manner to see if they can be improved.
5. Ensure that you have established roles and responsibilities for documents not stored in the primary TMF to assist in driving the TMF to an acceptable level of completeness.

[1] GCP INSPECTIONS METRICS REPORT METRICS PERIOD: 1st April 2018 to 31st March 2019 DATE OF ISSUE: 12th February 2021