MOSAIQ Real-World Testing Results
2022 Performance Year
Download results or visit MOSAIQ Real-World Testing
Performance Year: 2022
Plan Report ID Number: For DRUMMOND use
Developer Name: Elekta
Product Name(s): MOSAIQ
Version Number(s): 2.7, 2.81, 2.82, 2.83
Certified Health IT Product List (CHPL) Product Number(s):
| MOSAIQ 2.7 | 15.04.04.1420.MOSA.28.03.1.201231 |
| MOSAIQ 2.81 | 15.04.04.1420.MOSA.28.01.1.190625 |
| MOSAIQ 2.82 | 15.04.04.1420.MOSA.28.02.1.200413 |
| MOSAIQ 2.83 | 15.04.04.1420.MOSA.28.03.1.201231 |
Developer Real World Testing Plan Page URL: https://www.elekta.com/products/oncology-informatics/mosaiq-real-world-testing/
Developer Real World Testing Results Report Page URL: https://www.elekta.com/products/oncology-informatics/mosaiq-real-world-testing-results/
Changes to Original Plan
| Summary of Change | Reason | Impact |
|---|---|---|
| Elekta currently maintains 2015 CEHRT status for MOSAIQ v2.7. Elekta surveyed numerous providers and its internal database to identify providers utilizing MOSAIQ v2.7. Elekta was unable to identify any providers for Real World Testing who were utilizing MOSAIQ v2.7. | Elekta believes that given there are newer released versions of MOSAIQ, most, if not all, providers have upgraded. Elekta’s database identified all clients who had purchased MOSAIQ v2.7. Elekta surveyed these clients and discovered that all were no longer utilizing MOSAIQ v2.7 for various reasons (i.e. Upgrades, software changes). | Elekta was unable to conduct real world testing on a MOSAIQ v2.7, and as such was not able to confirm the viability of the certified functionality within MOSAIQ v2.7. |
| For Criterion §170.315(f)(1), the methodology used to collect data was initially planned to be the analysis of interface logs. After development work was planned, it was decided the best mechanism to collect this data was through SQL queries. | This decision was made as it would be the most efficient and accurate way to access the data for the sites selected. Utilizing SQL queries provided insight into the individual sites use. | Minimal impact as the data was still collected. |
| Elekta intended to begin conducting real world testing on March 1, 2022. Elekta began conducting real world testing on July 6, 2022. | Due to resource constraints and difficulty confirming dates/ times with participating sites, Elekta was unable to begin real world testing within timeframe initially specified. | Seemingly no impact, as Elekta was still able to conduct 6 sessions of real- world testing with participating sites and completed them prior to January 1, 2023. |
Summary of Testing Methods and Key Findings
To conduct real world testing, Elekta utilized multiple mechanisms, including the following:
- Running Existing Reports in MOSAIQ: Elekta asked participating sites to run an existing report, “Promoting Interoperability Report”, on their local version of MOSAIQ for all clinic providers.
- SQL Queries: Elekta asked participating clinics for access to their SQL servers to run multiple queries for various data extracts.
- Visual Inspection: Elekta imported CCDA’s with privacy flags and visually checked those users with privacy viewing capabilities were able to view a CCDA, and those without privacy viewing capabilities were not.
- Interface logs: Elekta inspected numerous interface logs to analyze the rate of use for various certified functionalities.
Elekta worked with participating sites to collect the appropriate data. Elekta selected a three- month timeframe, April 1st to June 30th, for data analysis. The SQL queries, Reports, and Interface logs analyzed this three- month timeframe only to avoid any data outliers.
Standards and Updates:
| Standards and Updates Version | CHPL ID | Standards |
|---|---|---|
| MOSAIQ 2.7 | 15.04.04.1420.MOSA.28.0 3.1.201231 | All standards versions are those specified in the Common Clinical Data Set |
| MOSAIQ 2.81 | 15.04.04.1420.MOSA.28.0 1.1.190625 | As above with the addition of NCPDP SCRIPT Version 2017071 |
| MOSAIQ 2.82 | 15.04.04.1420.MOSA.28.0 2.1.200413 | As above |
| MOSAIQ 2.83 | 15.04.04.1420.MOSA.28.0 3.1.201231 | As above |
Standards Updates:
| Standard and version | NCPDP SCRIPT Version 2017071 |
|---|---|
| Updated certification criteria and associated product | § 170.315(b)(3) Electronic prescribing MOSAIQ 2.81, 2.82, 2.82 |
| CHPL ID | 15.04.04.1420.MOSA.28.01.1.190625 15.04.04.1420.MOSA.28.02.1.200413 15.04.04.1420.MOSA.28.03.1.201231 |
| Method used for standard update | Required - Cures |
| Measure used to demonstrate conformance | ePrescribing |
| Customer notification | May 31, 2019 |
Care Setting(s)
Elekta conducted six sessions of real- world testing. All sessions were conducted with practices in radiation oncology and medical oncology settings. Elekta successfully tested the following versions of MOSAIQ:
- MOSAIQ v2.81
- MOSAIQ v2.82
- MOSAIQ v2.83
Metrics and Outcomes
Criterion: §170.315(h)(1) Receive health summary using DIRECT
| Relied Upon Software | Secure Exchange Solutions SES Direct |
|---|---|
| Measurement/ Metric | Log files were used to track the frequency of DIRECT to receive health information |
| Expected Outcome(s) | Demonstrates real-world use of DIRECT messaging to receive clinical summaries sent by referring providers. |
| Outcomes | The average number of received clinical summaries via DIRECT Messaging was ~193 messages. This number represents the average number of clinical summaries received for all participating sites during the period of April 1st-June 30th of 2022. Elekta believes the DIRECT Messaging functionality demonstrated fully compliant real- world use. |
Criterion: §170.315(b)(2) Create a single reconciled list of medications, medication allergies, or problems from received summary and active patient record
| Measurement/ Metric | Reports were used to calculate the number of times that medications, problems, and allergies were incorporated into a medical record compared to the number of referrals into a clinic |
|---|---|
| Expected Outcome | It is expected that a high percentage of new patients and incoming referrals with clinical summaries are incorporated into the new patient records. The test will report the frequency of use of MOSAIQ to receive, reconcile, and incorporate clinical summaries. |
| Outcomes | On average, providers had reconciled lists of medications, medication allergies, or problems from received summary and active patient record zero times. |
| Outcomes | The average number of received clinical summaries via DIRECT Messaging was ~193 messages. This number represents the average number of clinical summaries received for all participating sites during the period of April 1st-June 30th of 2022. Elekta believes the DIRECT Messaging functionality demonstrated fully compliant real- world use. |
| Challenges Encountered (if applicable) | Elekta utilized the Promoting Interoperability measure report within MOSAIQ. Providers were not asked during which 90- day period they collected their Promoting Interoperability data. Elekta assumes the 90- day period selected for all providers to run the report, April 1- June 30, was not the same period providers used in submission of their PI data to CMS. Alternatively, the providers surveyed may typically not use MOSAIQ for their referral workflow and may instead lean on additional software. Given this, Elekta believes the lack of data or reconciliation efforts is not representative of the reliability of the functionality but instead the use level. |
Criterion: §170.315(e)(1) View, download and transmit to a third party
| Relied Upon Software | Medfusion by Medfusion, Inc. |
|---|---|
| Measurement/ Metric | SQL Queries were used to track the frequency of patient views, downloads, and transmissions to third parties |
| Expected Outcome | Number of patient views, downloads, and transmissions |
| Outcomes | Elekta calculated the average rates, given the six available sites, for view, download, and transmit to a third party during the period of April 1st to June 30th. The averages for the participating sites were as follows
|
Criterion: §170.315(b)(8) Security tags - summary of care – receive
| Measurement/ Metric | SQL queries were used to ascertain the frequency of clinical summaries received with a privacy indicator. A test CCDA with a privacy indicator was imported by an a) unauthorized user and b) an authorized user. |
|---|---|
| Expected Outcome | Numbers that reflect the frequency of received clinical summaries with a privacy indicator and a positive test of restriction of viewing rights of a summary with a privacy indicator to authorized users. |
| Outcomes | Among the six surveyed sites, the average number of CCDA’s received with a security tag was zero, and therefore the number of positive tests of restriction of viewing rights was zero. |
| Challenges Encountered (if applicable) | Elekta believes this may not be a functionality commonly used by providers within MOSAIQ hence the lack of available data. Elekta believes the lack of data is not representative of the reliability of the functionality but instead the use level. |
Criterion: §170.315(b)(3) Electronic Prescribing
| Relied Upon Software | First DataBank Medknowledge Framework 4.2 Surescripts ePrescribing Dr. First EPCS Gold |
|---|---|
| Measurement/ Metric | Reports were used to determine the frequency of ePrescribing messages and the number of errors during transmission. |
| Expected Outcome | Clinicians will be able to manage outpatient prescriptions with local pharmacies with few errors. Less than 1% of errors are expected. |
| Outcomes | The e-prescription rate of success for transmission was 100%. On average providers transmitted ~1718 e-prescriptions, for which there were no errors. Therefore, the rate of failure of transmission was 0. |
Criterion: §170.315(f)(1) Transmission to Immunization Registries
| Measurement/ Metric | SQL queries were used to determine the use of the immunization interface message transmissions. |
|---|---|
| Expected Outcome | It is expected that clinics can transmit their immunization information successfully. Error rates will be tracked. |
| Outcomes | Providers on average did not utilize the immunization interface. The average number of immunizations sent was 0. |
| Challenges Encountered (if applicable) | Elekta believes that the immunization interface is not being used because Elekta users are predominantly focused on oncology care and do not typically send immunizations. Elekta believes the lack of data is not representative of the reliability of the functionality but instead the use level. |
Criterion: §170.315(g)(7) Application Access- Patient Selection
| Measurement/ Metric | Interface logs were reviewed to determine the number of API applications registered and the number of patient selection API transactions over time. |
|---|---|
| Expected Outcome | API applications will be able to receive properly-authorized patient ID’s. |
| Outcomes | Currently, there are no applications utilizing Elekta’s available API. Therefore, there have been no applications able to receive properly authorized patient ID’s. |
| Challenges Encountered (if applicable) | While Elekta’s API is available for use to any vendor, there has been little interest from application developers. Elekta believes this to be because Elekta is an oncology specific EHR and does not have a large margin of market share. To date, no applications have progressed beyond Elekta’s sandbox environment. Elekta believes that while the API is not being used, it is still functioning compliantly. |
Criterion: §170.315(g)(8) Application access- data category request
| Relied Upon Software | Carefluence |
|---|---|
| Measurement/ Metric | Interface logs were reviewed to measure the volume of patient laboratory results retrievals. |
| Expected Outcome | The patient lab results will be retrieved in full and without error. |
| Outcomes | No patient lab results were retrieved. Currently there are no applications utilizing our available API. Therefore, there have been no data calls to Elekta’s API. |
| Challenges Encountered (if applicable) | As mentioned above, there are currently no applications making use of Elekta’s API. Elekta believes this is because Elekta is an oncology specific EHR and does not have a large margin of market share. Elekta believes that while the API is not being used, it is still functioning compliantly. |
Criterion: §170.315(h)(1) Send health summary using DIRECT
| Relied Upon Software | Secure Exchange Solutions SES Direct |
|---|---|
| Measurement/ Metric | Log files were used to track the frequency of DIRECT to send health information. |
| Expected Outcome | It is expected that health care providers will receive clinical summaries using the DIRECT protocol. |
| Outcomes | The average number of clinical summaries sent via DIRECT Messaging was 11 messages. This number represents the average number of clinical summaries sent for all participating sites during the period of April 1st-June 30th of 2022. Elekta believes health care providers are compliantly receiving clinical summaries using the DIRECT protocol. |
Criterion: §170.315(b)(1) Transitions of Care
| Measurement/ Metric | SQL queries were used to determine the number of clinical summaries sent to providers. |
|---|---|
| Expected Outcome | It is expected that a high percentage of oncology treatment result in clinical summaries sent to referring providers. |
| Outcomes | Elekta utilized a SQL query and looked at data from April 1st – June 30th. During this three- month period, an average of 13 CCDAs were sent to a direct mail address. Elekta believes MOSAIQ is functioning compliantly given the general use of the functionality. |
Criterion: §170.315(g)(9) Application Access- all data request
| Relied Upon Software | Carefluence |
|---|---|
| Measurement/ Metric | Logs were used to measure the volume of all data requests |
| Expected Outcome | API applications will be able to request and receive patient’s properly authorized clinical summary data. Error rates will be tracked. |
| Outcomes | Currently, Elekta has no applications utilizing the available API. Therefore, no API applications were able to request and receive patient’s properly authorized clinical summary data. |
| Challenges Encountered (if applicable) | As mentioned above, there are currently no applications making use of Elekta’s API. Elekta believes this is because Elekta is an oncology specific EHR and does not have a large margin of market share. Elekta believes that while the API is not being used, it is still functioning compliantly. |
Criterion: §170.315(b)(7) Security tags- summary of care- send
| Measurement/ Metric | Test CCDA’s with a privacy indicator were created by a) an unauthorized user and b) an authorized user and visually analyzed. |
|---|---|
| Expected Outcome | All CCDA’s designated as “private” by the user will be constrained with the Confidentiality Code in accordance with the standard as specified in DS4P R1. |
| Outcomes | For all participating sites, CCDA’s with a privacy indicator were imported into the site’s MOSAIQ. Users without privacy viewing rights were unable to see the contents of the CCDA. Users with privacy viewing rights were able to see and import the contents of the CCDA into the patient chart. |
Criterion: §170.315(b)(6) Data export
| Measurement/ Metric | SQL queries were used to measure the number of batch export instances and CCDA’s per batch. |
|---|---|
| Expected Outcome | Users will be able to batch export clinical summaries for multiple patients for use by external systems. Error rates will be tracked. |
| Outcomes | Among the six surveyed sites, the average number of batch export instances was ~2379. The average number of batch export errors was ~13. |
| Challenges Encountered (if applicable) | For two sites, we were unable to report on their SQL data for this specific measure. The data produced contained errors. Elekta assumes the data for the other four available sites to be free from error and therefore reliable. |
Key Milestones
| Key Milestone | Care Setting | Date/ Timeframe |
|---|---|---|
| Finalization of SQL Queries for Real World Testing | June 15, 2022 | |
| Real World Testing Conducted with Site #1 | Radiation Oncology and Medical Oncology | July 6, 2022 |
| Real World Testing Conducted with Site #2 | Radiation Oncology and Medical Oncology | July 26, 2022 |
| Real World Testing Conducted with Site #3 | Radiation Oncology and Medical Oncology | July 28, 2022 |
| Real World Testing Conducted with Site #4 | Radiation Oncology and Medical Oncology | August 4, 2022 |
| Real World Testing Conducted with Site #5 | Radiation Oncology and Medical Oncology | September 15, 2022 |
| Real World Testing Conducted with Site #6 | Radiation Oncology and Medical Oncology | November 17, 2022 |
| Final Results collected and aggregated | November 17, 2022 |
Attestation
The Real World Testing Results above are complete with all required elements. All information included in these results are up to date and fully address the health IT developer’s Real World Testing requirements.
Authorized Representative: Ken Hotz
Authorized Representative Email: interop@elekta.com
Authorized Representative Phone: +1 408 306 7744
Authorized Representative Signature: Ken Hotz
Date: March 14, 2022