HIMSS and RSNA
Integrating the Healthcare Enterprise

 

 IHE/MESA Report Manager Tests

 

Electronic Radiology Laboratory
Mallinckrodt Institute of Radiology
510 South Kingshighway Blvd.
St. Louis, MO  63110
314.362.6965 (Voice)
314.362.6971 (Fax)

 

Revision 10.0.0
15-August-2006

Contents:

Report Manager Tests
Introduction
Message Attributes
Message Values
Configuration
Starting the MESA Servers
Resetting the MESA Servers
Test Instructions

  1. Report Manager Test 601: Simple Imaging Report: No Ref
  2. Report Manager Test 652: Add a Section
  3. Report Manager Test 653: Remove a Reference
  4. Report Manager Test 654: Change Text
  5. Report Manager Test 661: Simple Report Export
  6. Report Manager Test 663: Report Export with Two Image References
  7. Report Manager Test 671: Finalize Numeric Report

Reporting Workflow Tests

  1. Report Manager Test 1601: Interpretation Worklist
  2. Report Manager Test 1603: RWF Use Case 2: Workitem Deferred

Basic Security Tests

  1. Report Manager Test 1511: Simple Imaging Report

Individual Tests for Cardiology

  1. Report Manager Test 20501: Verify HL7 Encapsulated Report Message Content (CARD-7)
  2. Test Case 20502:  Visual Verify of PDF Content (CARD-7)
  3. Test Case 20503: Receive HL7 Encapsulated Report (CARD-7)
  4. Test Case 20511:  DRPT: Describe Report Manager Methods
  5. Report  Manager Test 20520: Verify HL7 Report Reference Submission Message Content and Retrieval (CARD-8)
  6. Report  Manager Test 20530: Verify DICOM Encapsulated Report Message Content (CARD-9)     

Report Manager Tests

Introduction

Integration Profiles and Test Procedures

This document lists a number of tests for Report Manager Systems. You may not be responsible for all of these tests.
Please refer to the Connectathon web tool to list the required tests for your system. The web address of this tool depends on the year and project manager. Please contact the appropriate project manager to obtain this information.

Message Attributes

DICOM SR Objects and HL7 ORU messages contain attributes that are not used or required by the IHE Technical Framework: Year 3.  This section contains two tables defining those attributes that will be evaluated in the Report Manager tests.  Other values may be required by the Technical Framework but not evaluated with this software.
In the tables below, the columns marked Value and Present indicate the scope of the evaluation for these messages.  Fields that are marked in the Value column indicate the tests will expect a specific value in the field (for example, we specify the patient name).  Fields marked in the Present column are tested to see if the field contains a value, but the tests cannot evaluate for a specific value.

Tag

Attribute Name

Value

Present

0008 0016

SOP Class UID

x

 

0008 0050

Accession Number

x

 

0008 0060

Modality

x

 

0010 0010

Patient Name

x

 

0010 0020

Patient ID

x

 

0020 000D

Study Instance UID

x

 

0040 A040

Value Type

x

 

0040 A043

Concept Name Code Sequence (Report Title)

 

 

> 0008 0100

Code Value

x

 

> 0008 0102

Coding Scheme Designator

x

 

> 0008 0104

Code Meaning

x

 

0040 A360

Predecessor Documents Sequence

 

 

> 0020 000D

Study Instance UID

x

 

> 0008 1115

Referenced Series Sequence

 

 

>> 0020 000E

Series Instance UID

x

 

>>> 0008 1150

Referenced SOP Class UID

x

 

>>> 0008 1155

Referenced SOP Instance UID

x

 

0040 A491

Completion Flag

x

 

0040 A493

Verification Flag

x

 

0040 A504

Content Template Sequence

 

 

> 0008 0101

Mapping Resource

x

 

> 0040 DB00

Template Identifier

x

 

0040 A730

Content Sequence

 

 

> 0040 A010

Rerlationship Type

x

 

> 0040 A040

Relationship Value

x

 

Other attributes

As needed for the Relationship Value

x

 

Message Values

Configuration

The Report Manager scripts described below use an ASCII configuration file to identify parameters such as host names and port numbers.  The configuration file is named rptmgr_test.cfg and is included in the directory $MESA_TARGET/mesa_tests/rad/actors/rpt_mgr.  Edit the file and change entries (host name, port number) which pertain to your system.  Your system is identified by entries that begin with TEST. The table below shows the configuration file as shipped with the MESA software.


Variable

Value

TEST_HL7_HOST

rpt_manager

TEST_HL7_PORT

2750

TEST_MPPS_HOST

rpt_manager

TEST_MPPS_PORT

2700

TEST_MPPS_AE

RPTMGR_PPS

TEST_CSTORE_AE

REPORT_MANAGER

For IHE Basic Security tests, all messages are exchanged using TLS. MESA servers are run on the same ports but with the TLS option. The configuration file that identifies your information is rptmgr_secure.cfg. This separate file allows you to use different port numbers for your secure and standard configurations. You may decide to use the same port numbers for both types of communication. The MESA software will only use all secure or all standard communication for a test; we do not mix communication protocols.
The file $MESA_TARGET/runtime/rpt_repos/ds_dcm.cfg is used to configure the MESA Report Repository.  The only parameter users should change is the LOG_LEVEL value.  Log levels are defined in section 1.5.  DICOM configuration parameters are listed in the table below.

Application

AE Title

Port

MESA Report Repository

REPORT_ARCHIVE

2800

MESA Audit Record Repository

none

4000

MESA Enterprise Report Repository

None

3300

The MESA Enterprise Report Repository is configred to receive HL7 messages (reports) on port 3000.
Read the Runtime Notes section of the Installation Guide to determine the proper settings for the MESA runtime environment.

Starting the MESA Servers

These instructions assume you are using a terminal emulator on Unix systems or an MS DOS command window under Windows 2000 or XP.  Each test uses a command line interface; there is no graphical user interface. Before you start the test procedure, you need to start the MESA Report Repository and MESA Enterprise Report Repository servers.  Make sure the appropriate database is running (PostgreSQL, SQL Server).  To start the MESA servers:

  1. Enter the Report Manager exam fileder: mesa_tests/rad/actors/rpt_mgr
  2. Execute the appropriate script to start the servers:
     scripts/start_mesa_servers.csh [loglevel]  (Unix)
     scripts\start_mesa_servers.bat [loglevel] (Windows)
Loglevel is an optional argument (0 is default).  Log levels are:

  1. no logging
  2. errors
  3. warnings
  4. verbose
  5. conversational (really verbose)

The instructions above set the log level for the MESA Enterprise Report Repository.  The MESA Report Repository is a DICOM server and uses a different mechanism to set the log level.  Edit the configuration file $MESA_TARGET/runtime/rpt_repos/ds_dcm.cfg and change the variable LOG_LEVEL.
When you are finished running one or more tests, you can stop the servers: 

     scripts/stop_mesa_servers.csh  (Unix)
     scripts\stop_mesa_servers.bat  (Windows)
Log files are stored in $MESA_TARGET/logs.
For the security tests, the MESA servers are started with different scripts. These are scripts/start_mesa_secure.csh and scripts\start_mesa_secure.bat. The log levels are the same as for the standard tests. The MESA servers are stopped using these scripts: scripts/stop_mesa_secure.csh and scripts\stop_mesa_secure.bat.

Resetting the MESA Servers

You can store as many reports in the MESA Report Repository and MESA Enterprise Report Repository as you wish.  You might want to clear the repositories before you start tests or after a few rounds.  To clear the repository: 

     perl  scripts/clear_report_repository.pl

 Test Instructions

Report Manager Test 601: Simple Imaging Report: No Ref

In this test, a MESA Report Creator sends an unverified report to the Report Manager.  The Report Manager creates a verified report and sends the verified report to the MESA Report Repository.

References

Instructions

  1. The patient name used for this report is CRTHREE^PAUL.
  2. Send the unverified report to your Report Manager: 
         perl 601/load_601.pl
  3. Start the MESA repositories as described in Starting the MESA Servers.
  4. Clear the MESA repository: 
         perl  scripts/clear_report_repository.pl
  5. Create a verified report and send to the MESA Report Repository.

Evaluation

  1. Evaluate your verified report: 
     perl  601/eval_601.pl

Supplemental Information

To be more explicit, we expect you to do this:

  1. Copy the original report.  Keep the original structure and attributes.
  2. Set the Verification Flag (0040 A493) to VERIFIED
  3. Add (0040 A073) Verifying Observer Sequence (see table below for values)
  4. Add (0040 A360) Predecessor Documents Sequence

Attribute Name

Tag

Value

Verifying Observer Sequence

0040 A073

 

> Verifying Observer Name

0040 A075

PERRY^JOHN

> Verifying Observer Identification Code Sequence

0040 A088

 

>> Code Value

0008 0100

6060

>> Coding Scheme Designator

0008 0102

IHEDEMO

>> Code Meaning

0008 0104

PERRY^JOHN

> Verifying Organization

0040 A027

Midwest Aviators

> Verification DateTime

0040 A030

Your Date/Time

Report Manager Test 652: Add a Section

In this test, a MESA Report Creator sends an unverified report to the Report Manager. The Report Manager adds a section to the end of the report (121078: Addendum) and verifies the report. This verified report with an additional section is sent to the MESA Report Repository.

References

Instructions

  1. The patient name for this report is CTFIVE^JIM.
  2. Send the unverified report to your Report Manager: 
         perl 652/load_652.pl
  3. Start the MESA repositories as described in Starting the MESA Servers.
  4. Clear the MESA repository: 
         perl  scripts/clear_report_repository.pl
  5. Create a verified report and send to the MESA Report Repository.  Make sure to add the Addendum after the Conclusions section of the original report.

Evaluation

  1. Evaluate your verified report: 
     perl  652/eval_652.pl

Supplemental Information

Report Manager Test 653: Remove a Reference

In this test, a MESA Report Creator sends an unverified report to the Report Manager with references to two images.  The Report Manager removes one of the references and verifies the report.  This verified report with an additional section is sent to the MESA Report Repository.

References

Instructions

  1. Send the unverified report to your Report Manager:
     perl 653/load_653.pl
The patient name for this report is MRTHREE^STEVE.

  1. Start the MESA repositories as described in section 1.5 above.
  2. Clear the MESA repository: 
    3. perl  scripts/clear_report_repository.pl
  3. Create a verified report and send to the MESA Report Repository.  Remove the second image reference in the report.

Evaluation

  1. Evaluate your verified report: 
     perl 653/eval_653.pl

Supplemental Information

Report Manager Test 654: Change Text

In this test, a MESA Report Creator sends an unverified report to the Report Manager.  The Report Manager modifies text in the report and verifies the report.  This verified report with an modified text is sent to the MESA Report Repository.

References

Instructions

  1. The patient name for this test is CTSIX^JIM.
  2. Send the unverified report to your Report Manager: 
         perl 654/load_654.pl
  3. Start the MESA repositories as described in section 1.5 above.
  4. Clear the MESA repository: 
    perl  scripts/clear_report_repository.pl
  5. Create a verified report and send to the MESA Report Repository. Change the text in the Conclusions section to read: “Modified conclusions.”

Evaluation

  1. Evaluate your verified report: 
     perl 654/eval_654.pl

Supplemental Information

Report Manager Test 661: Simple Report Export

In this test, the Report Manager receives an SR object and needs to export an HL7 Report to the MESA Enterprise Report Repository (port 3300).  This test will reuse SR objects from earlier tests, so you might want to clear your Report Manager.

References

Instructions

  1. Send the unverified report to your Report Manager.  This is the same report as used for the 601 test:
     perl 661/load_661.pl
The patient name for this report is CRTHREE^PAUL.

  1. Start the MESA repositories as described in section 1.5 above.
  2. Clear the MESA repository: 
     perl  scripts/clear_report_repository.pl
  1. Create a verified report and export the verified report in HL7 format to the MESA Enterprise Report Repository (port 3300).

Evaluation

  1. Evaluate your verified report: 
     perl 661/eval_661.pl

Supplemental Information

For this test, the MESA tools evaluate the SOP Instance UID of the original SR document against the SOP Instance UID of the final document that  you create.  This will result in a “failure”.  You can ignore this failure.

Report Manager Test 663: Report Export with Two Image References

This test is similar to test 661.  The input report includes two image references.  The test data comes from test 653.

References

Instructions

  1.  The patient name fot this report is MRTHREE^STEVE.
  2. Send the unverified report to your Report Manager.  This is the same report as used for the 653 test: 
         perl 663/load_663.pl
  3. Start the MESA repositories as described in Starting the MESA Servers.
  4. Clear the MESA repository: 
         perl  scripts/clear_report_repository.pl
  5. Create a verified report and export the verified report in HL7 format to the MESA Enterprise Report Repository (port 3300).

Evaluation

  1. Evaluate your verified report: 
     perl 663/eval_663.pl

Supplemental Information

For this test, the MESA tools evaluate the SOP Instance UID of the original SR document against the SOP Instance UID of the final document that  you create.  This will result in a “failure”.  You can ignore this failure.

Report Manager Test 671: Finalize Numeric Report

In this test, a MESA Report Creator sends an unverified, numeric report to the Report Manager.  The Report Manager creates a verified report and sends the verified report to the MESA Report Repository.

References

Instructions

  1. The patient name for this report is CRTHREE^PAUL.
  2. Send the unverified report to your Report Manager: 
         perl 671/load_671.pl
  3. Start the MESA repositories as described in Starting the MESA Servers..
  4. Clear the MESA repository: 
         perl  scripts/clear_report_repository.pl
  5. Create a verified report and send to the MESA Report Repository.

Evaluation

  1. Evaluate your verified report: 
     perl 671/eval_671.pl

Supplemental Information

Reporting Workflow Tests

This section describes tests that are specific to the IHE Reporting Workflow integration profile.

Report Manager Test 1601: Interpretation Worklist

Test 1601 covers the Interpretation task in the Reporting Workflow integration profile (see IHE TF, Vol I: section 13.3). This test only covers the interpretation step and does not include creation of any other items (such as transcriptions or reports). The Report Manager is asked to create an Interpretation workitem for this patient. The workitem should be scheduled for station class code REPORTER, station name code REPORTER_1 and scheduled human performer code RADIOLOGIST^JOE.

References

Instructions

The nominal patient name is WASHINGTON^GEORGE.
To change patient name, patient ID prior to the test: 

     cd $MESA_TARGET/mesa_tests/rad/msgs
     perl mesa_reset/1601.pl ihe-us/1601.var  (or)
     perl mesa_reset/1601.pl

Standalone Report Managers

Standalone Report Managers should be configured to send Performed Work Status messages to the MESA Image Manager.
To run this test: 

     perl scripts/rptmgr_rwf.pl 1601 <log>

Evaluation

To evaluate this test: 

     perl 1601/eval_1601_standalone.pl <log>  <GPPPS AE title>

Report Manager Grouped with Order Filler

Report Manager Grouped with Image Manager

Report Manager Test 1603: RWF Use Case 2: Workitem Deferred

Test 1603 covers the Interpretation task in the Reporting Workflow integration profile (see IHE TF, Vol I: section 13.4.2). In this test, the Report Creator claims a workitem and then defers it by setting the status back to “SCHEDULED”. The Report Manager under test will be evaluated by examining reponses to three separate worklist queries:

The Report Manager is asked to create an Interpretation workitem for this patient. The workitem should be scheduled for station class code REPORTER, station name code REPORTER_1 and scheduled human performer code RADIOLOGIST^JOE.

References

Instructions

The nominal patient name is LINCOLN^ABRAHAM.
To change patient name, patient ID prior to the test: 

     cd $MESA_TARGET/mesa_tests/rad/msgs
     perl mesa_reset/1603.pl ihe-us/1603.var  (or)
     perl mesa_reset/1603.pl

Standalone Report Managers

Standalone Report Managers should be configured to send Performed Work Status messages to the MESA Image Manager.
To run this test: 

     perl scripts/rptmgr_rwf.pl 1603 <log>

Evaluation

To evaluate this test: 

     perl 1603/eval_1603_standalone.pl <log>  <GPPPS AE title>

Report Manager Grouped with Order Filler

Report Manager Grouped with Image Manager

Basic Security Tests

This section describes tests that are specific to the IHE Basic Security integration profile. If you have the MESA servers running for the “standard” tests, you should stop those servers now. You will need to start the MESA secure servers with a different script.

Report Manager Test 1511: Simple Imaging Report

Report Manager Test 1511 uses the same structure as test 601. The Report Manager is expected to communicate with other systems using TLS negotiation and to send appropriate audit messages to the MESA syslog server.
The table below lists the Audit Messages that should be generated by your Report Manager. Please refer to the document IHE Tests: Transaction Sequences for the full context of these messages. You might trigger other messages to the Audit Record Repository based on your interaction with your Report Manager.

Identifier

Description

Source

Destination

1591.012

DICOM-instances-used

Report Manager

Audit Record Repos

1591.014

Begin-storing-instances

Report Manager

Audit Record Repos

In this test, a MESA Report Creator sends an unverified report to the Report Manager.  The Report Manager creates a verified report and sends the verified report to the MESA Report Repository.

References

Instructions

  1. Send the unverified report to your Report Manager. This is the same report used for test 601, but the script sends the data using TLS communication.
     perl  1511/load_1511.pl
The patient name for this report is CRTHREE^PAUL.

  1. Start the secure MESA servers as described in section 1.5 above.
  2. Clear the MESA repository: 
     perl scripts/clear_report_repository.pl
  1. Create a verified report and send to the MESA Report Repository.

Evaluation

  1. Evaluate your verified report: 
     perl 1511/eval_1511.pl

Supplemental Information

More explicit instructions about the content of the modified report are given in section 2.1 above.

Individual Tests for Cardiology

Report Manager Test 20501: Verify HL7 Encapsulated Report Message Content (CARD-7)

Test 20501 tests the creation and content of an HL7 message with an encapsulated PDF report.
In this test, the Report Manager needs to export an HL7 Report to the MESA Enterprise Report Repository. 

References

Instructions

To run this test, follow these steps using a DOS window or terminal emulator:

  1. Set the current directory to $MESA_TARGET/mesa_tests/card/actors/rpt_mgr.
  2. Make sure the MESA servers have been started as described in Starting the MESA Servers.
  3. Construct an ORU message with an encapsulated PDF report. Send the message to the MESA Enterprise Report Repository (port 3300).
  4. Run the evaluation script below.

Evaluation

  1. Run the evaluation script
     perl  20501/eval_20501.pl <log level>
  1. The evaluation output is found in 20501/grade_20501.txt. The final result should indicate 0 errors. Submit the result run at log level 4 to the Project Manager.

Supplemental Information

For this test, the MESA tools evaluate the following elements:
•           OBR25 = R/P/F/C
For OBX w/ Study Instance UID, verify:
•           OBX3= “113014/DICOM Study/DCM”
•           OBX5 = valid format for SIU
•           OBX11 = “O”
For OBX w/ PDF report, verify:
•           OBX3 = a (any) Report Title, e.g., a Report Title from Vol 2 Appendix D
•           OBX 5 Source Application ID = a valid ISO OID
•           OBX5 Type of Data = “Application”
•           OBX5 Data Subtype = “PDF”
•           OBX11= R/P/F/C and matches OBR25
Any missing or incorrect elements will result in a “failure”.  You cannot ignore this failure.

Test Case 20502:  Visual Verify of PDF Content (CARD-7)

Test 20502 is the simple visual verification of the PDF content.  This test tests the case where the PDF becomes disassociated with the PDF message content.
In this test there is no MESA software or scripts involved.  Any patient name and ID can be used. 

References

Instructions

Using a “good” report within your Report Manager, the vendor should create a word or pdf file using the document naming convention of: CompanyName_Product_ 20502_RM_2005.doc .  Using any vendor report viewing tool cut and paste in information from your own product which demonstrates that the followng data elements are visible in a report:
•           Identity of signing clinician (legal signature)
The following items are recommended, but not required:
•           Patient name
•           Patient ID
•           Date of Procedure
•           Type of Procedure
•           Date of Report
•           Status of Report

Evaluation

Submit the file created above to the Project Manager. In the file, note which of the items listed above (Patient name, Patient ID, etc.) are included).

Supplemental Information

Test Case 20503: Receive HL7 Encapsulated Report (CARD-7)

Test 20503 tests the ability to receive an HL7 message with an encapsulated PDF report.
To run this test, follow these steps using a DOS window or terminal emulator:

References

Instructions

  1. Set the current directory to $MESA_TARGET/mesa_tests/card/actors/rpt_mgr.
  2. Make sure the MESA servers have been started, if not run the following:
     scripts/start_mesa_servers.csh  (Unix)
     scripts\start_mesa_servers.bat  (Windows)
When you are finished running one or more tests, you can stop the servers: 
     scripts/stop_mesa_servers.csh  (Unix)
     scripts\stop_mesa_servers.bat  (Windows)
  1. Run the test script as follows:
     perl  20503/20503_rpt_mgr.pl

Evaluation

  1. There  is no evaluation script for the test. The vendor should create a word or pdf file using the document naming convention of: CompanyName_Product_ 20503_ERR_2005.doc .  Using any vendor tool cut and paste in information from your own product which demonstrates that the report was successfully stored in your product.  A query/retrieve screen snapshot of that report or a database report screen snapshot would be a good example. Submit this result to the Product Manager.

Supplemental Information

Test Case 20511:  DRPT: Describe Report Manager Methods

Do not divulge any proprietary information.  Create a Word document with the following naming convention:    CompanyName_Product_20511_RM_2005.doc and send it to the Technical Project Manager.

References

Instructions

In 500 words or less for each, describe the following for the Report Manager’s:

  1. “RELEASE MECHANISM”, eg., are reports of all statuses automatically sent to the ERR?  Is manual interventional always necessary? Etc.
  2. If the Report Manager is not grouped with the Report Repository, describe the method to change the web access point of the Report Repository (used in CARD-8)
  3. Have you thought about a method to change the URLs if the web addresses change? (simply answer “yes” or “no”)
  4. Describe the Report Signature and Verification process

Evaluation

Supplemental Information

Report  Manager Test 20520: Verify HL7 Report Reference Submission Message Content and Retrieval (CARD-8)

Test 20520 tests the creation and content of an HL7 message with a referenced PDF report.
This test will require your system to create a HL7 ORU(encapsulated report by reference - unsoliticited ) according to the IHE Cardiology Techinical Framework Year2:2005-2006 Volume II, Section 4.7.

References

Instructions

  1. Start the MESA servers as instructed in section 1.5.
  2. Clear existing reports from the MESA Report Manager: 
         perl  scripts/reset_rpt_mgr.pl
  3. Run the test to send the ORU message to MESA Enterprise Report Repository (port 3300).
     perl scripts/rptmgr_drpt.pl 20520  <log level>

Evaluation

  1. Evaluate your ORU message:
     perl  20520/eval_20520.pl <log level>
  1. The grade file will be found in 20520/grade_20520.txt. Submit the grade file created at log level 4 to the Project Manager.
  2. The MESA script will extract the URL from OBX segment and print in the grade file.  The user should cut and paste the URL into a browser and manually perform a query on the report link, retrieve, and view the report.   The vendor should create a word file using the document naming convention of: CompanyName_Product_ 20520_RM_2005.doc .  The vendor should cut and paste in a screen snapshot which demonstrates that the report was successfully displayed.
  3. Send the file with other results to the Technical Project Manager.

Supplemental Information

Report  Manager Test 20530: Verify DICOM Encapsulated Report Message Content (CARD-9)

Test 20530 tests the creation and content of a DICOM message with an encapsulated PDF report.
MESA tools will act as a Report Creator by sending an encapsulated PDF report to test Report Manager.

References

Instructions

  1. Start the MESA servers as instructed in Starting the MESA Servers.
  2. Clear existing reports from the MESA Report Manager:
     perl  scripts/reset_rpt_mgr.pl
  1. Run the test to accept MESA generated PDF report and send the generated DICOM message to MESA Report Repository (port 2800).
     perl  scripts/rptmgr_drpt.pl 20530 <log level>

Evaluation

  1. Evaluate your DICOM message content:
     perl  20530/eval_20530.pl <log level>

Supplemental Information

For this test, the MESA tools evaluate the following elements:

Any missing or incorrect elements will result in a “failure”.  You cannot ignore this failure.