HIMSS and RSNA
Integrating the Healthcare Enterprise

 

 IHE/MESA Evidence Creator Tests

 

Electronic Radiology Laboratory
Mallinckrodt Institute of Radiology
510 South Kingshighway Blvd.
St. Louis, MO  63110
314.362.6965 (Voice)
314.362.6971 (Fax)

 

Revision 10.15.0
5-Apr-2007


Contents

Evidence Creator Tests
Introduction
Message Attributes
Message Values
Configuration
Starting the MESA Servers
Submission of Results
Individual Tests

  1. 251: Storage Commitment Assoc Negotiation
  2. 281: Example Images
  3. 283: DICOM Composite Object Evaluation - DVT
  4. 284: DICOM Composite Object Evaluation - DICOM3TOOLS
  5. 301: Evidence Creator Storage and Commitment
  6. 311: Create/Render Encapsulated PDF
  7. Evidence Creator Test 500: Display Calibration
  8. Evidence Creator Test 511: Key Image Note 511
  9. Evidence Creator Test 512: Key Image Note 512
  10. Evidence Creator Test 513: Key Image Note 513
  11. Evidence Creator Test 521: Consistent Presentation of Images
  12. Evidence Creator Test 551: Example Image/GSPS/BSPS and Spatial Registration Objects.
  13. Evidence Creator Test 552: Example Key Image Note
  14. Evidence Creator Test 1412: PWF CT
  15. Evidence Creator Test 1700: Evidence Document Description
  16. Evidence Creator Test 1701: Evidence Document Management in Scheduled Workflow
  17. Test 283: DICOM Composite Object Evaluation - DVT
Nuclear Medicine Specific Tests
  1. 2800: Result Screen Export Documentation
  2. 2801: NM Reconstructed Images Special Requirements
  3. 2802: NM Reconstructed Images Cardiac Views
  4. 2803: NM Result Export
  5. 2804: NM Result Export Screen 1
Cardiology Specific Tests
  1. Evidence Creator Test 20640: Evidence Creation – Cath –Vendor Interoperability
  2. Evidence Creator Test 20641: Evidence Creation – Echo –Vendor Interoperability
  3. Evidence Creator Test 20700: Stress Profile - SOP Class Support
  4. Evidence Creator Test 20740: Stress Profile - Sample Datasets –Vendor Interoperability
Image Fusion (FUS) Profile Tests
  1. Evidence Creator Test 3510: Create and Store Spatial Registration – same FoR
  2. Evidence Creator Test 3512: Create and Store Spatial Registration – different FoR
  3. Evidence Creator Test 3514: Create BSPS with Spatial Registration
  4. Evidence Creator Test 3516: Modify Existing Spatial Registration
  5. Evidence Creator Test 3540: Create and Store BSPS IOD – same FoR
  6. Evidence Creator Test 3541: Modify Existing BSPS

Mammography Image Integration (MAMMO) Profile Tests

  1. Evidence Creator Test 3916: Sample Mammo CAD SR Objects - Vendor Interoperability

Evidence Creator Tests

Introduction

This document describes several tests for Evidence Creator systems.  The Display Consistency tests are defined in a separate document: Display Consistency Test Plan for Image Creator.

Integration Profiles and Test Procedures

This document lists a number of tests for Evidence Creator systems. You may not be responsible for all of these tests.
Please refer to the Connectathon web tool to list the required tests for your system. The web address of this tool depends on the year and project manager. Please contact the appropriate project manager to obtain this information.

Message Attributes

Message Values

Configuration

The MESA Image Manager maintains a database of DICOM applications used for C-Move operations.  Add an entry for the storage SCP(s) associated with your workstation.  Edit the text file $MESA_TARGET/db/loaddicomapp.pgsql (Unix) or $MESA_TARGET/db/loaddicomapp.sql (Windows NT)  Use the existing entries as a template and add entries for your workstations as appropriate.  The column names found in the SQL insert statements are described in the following table.

Column Name

Description

aet

DICOM Application Entity Title.  Must be unique.

host

Host name (or IP address) of the application.

port

TCP/IP port number for receiving associations.

org

The organization that operates the device.  Useful if multiple organizations use the Image Manager.

com

A comment field.

You can test your work as follows:
            perl load_apps.pl imgmgr
The file $MESA_TARGET/runtime/imgmgr/ds_dcm.cfg is used to configure the MESA Image Manager.  The only parameter users should change is the LOG_LEVEL value.  Log levels are defined in section 1.5.  DICOM configuration parameters are listed in the table below.

Application

AE Title

Port

MESA Image Manager

MESA_IMG_MGR

2350

Read the Runtime Notes section of the Installation Guide to determine the proper settings for the MESA runtime environment.

Starting the MESA Servers

These instructions assume you are using a terminal emulator on Unix systems or an MS-DOS command window under Windows 2000 or XP. Each test uses a command line interface; there is no graphical user interface. Before you start the test procedure, you need to start the MESA Image Manager servers. Make sure the appropriate database is running (PostgreSQL, SQL Server). To start the MESA servers:

  1. Enter the Image Display exam folder: $MESA_TARGET/mesa_tests/rad/actors/imgcrt or $MESA_TARGET/mesa_tests/rad/actors/evdcrt
  2. Execute the appropriate script to start the servers:
      scripts/start_mesa_servers.csh  (Unix)
      scripts\start_mesa_servers.bat  (Windows)

Log levels are set for the MESA Image Manager in the file: $MESA_TARGET/runtime/imgmgr/ds_dcm.cfg.  Log levels are:

  1. no logging
  2. errors
  3. warnings
  4. verbose
  5. conversational (really verbose)

When you are finished running one or more tests, you can stop the servers:

      scripts/stop_mesa_servers.csh  (Unix)
      scripts\stop_mesa_servers.bat  (Windows)

Log files are stored in $MESA_TARGET/logs.

Starting Servers for Test 1701

The Test 1701 scripts are found in a different directory from the other Evidence Creator scripts.  The Test 1701 scripts are in the evdcrt (evidence creator) directory, and use a slightly different server control procedures. 
Before you start the test procedure, you need to start the MESA Image Manager servers.  Make sure the appropriate database is running (PostgreSQL, SQL Server).  To start the MESA servers:

  1. Enter the Evidence Creator folder: $MESA_TARGET/mesa_tests/rad/actors/evdcrt
  2. Execute the appropriate script to start the servers:
     perl scripts/mesa_servers.pl start 
Log levels are as described in the section above.
When you are finished running one or more tests, you can stop the servers:
     perl  scripts/mesa_servers.pl stop
Log files are stored in $MESA_TARGET/logs.

Submission of Results

Test descriptions below inform the reader to “submit results to the Project Manager”. This is does not mean “email”. The current submission process should be documented by the Project Manager, but will not include emailing files directly to the Project Manager.

Individual Tests

 

251: Storage Commitment Association Negotiation

This is a test of association negotiation with your Evidence Creator. An Image Manager that wants to send Storage Commitment N-Event reports will initiate a DICOM association with your modality and should propose to be an SCP of the Storage Commitment SOP Class (Push Model).

In this test, one association will be proposed. The MESA Image Manager proposes the SCP role. Your Evidence Creator should accept this association and proposed presentation context.

References

Instructions

  1. Start your server process that accepts Image Manager Storage Commitment association requests.
  2. Run the evaluation script for this test. This script sends the appropriate association requests and records results in 251/grade_251.txt:
         perl 251/eval_251.pl [-v]

Evaluation

  1. Submit the file 251/grade_251.txt generated above to the Project Manager. This file should indicate 0 errors for success.

Supplemental Information

281: Example Images

Test 281 is used to collect sample images, Key Object Notes, Evidence Documents and/or other DICOM composite objects produced by an Evidence Creator. The intent of the test is to send DICOM composite objects (DICOM Part 10 format) to the Project Manager for redistribution to other participants. This will allow them time to test/examine your data before an in-person meeting.

This test differs from normal tests in that you submit samples to the Project Manager so other participants can review those samples. Because of this, please submit your samples 2 WEEKS before the normal deadlines. This will give the other systems a chance to review your data.

References

Instructions

  1. Produce DICOM Part 10 files for the types of Evidence Documents your system system produces. If you produce more than one type of Evidence Document, produce at least one DICOM Part 10 file for each document type.
  2. Please the Part 10 files in a folder with a README.TXT file that describe each file. Use subfolders if that helps to organize your data.
  3. Place the entire folder in a zip file which is named 281_Your_Sytem_Name.zip
  4. If the file is small enough, you may upload it into the Connectathon webtool as results for this test. If the file is too large, send an email to the Project Manager stating that you are ready to submit results. Do not email the zipped objects. You will receive a response containing instructions for submitting your objects to an ftp site or sending in a CD. 281

Evaluation

Evaluation of this test occurs when other participants review your data. In the event that other participants find errors/issues, you may be asked to modify your data.

Supplemental Information

282: Example GSPS Objects

In this “test”, you use the MESA tools to collect GSPS object and then send those objects to the Project Manager for distribution to other vendors.  These are the objects that you would expect to use in the demonstration.  The MWL entries should have the codes used for the demonstration, but these are not available with this release.  We will use the MWL entry for the simple 211 case.
For modalities that produce GSPS objects, you should combine these instructions with those of Test 281 and send a set of images and corresponding GSPS objects.
Some modalities qualify for the CPI Integration Profile and do not produce GSPS objects.  These are the modalities that produce images already in P-values.  In that case, this test (282) is not necessary (unless you produce GSPS objects for other reasons).

References

Instructions

Evaluation

Supplemental Information

283: DICOM Composite Object Evaluation – DVT

In this test, you evaluate samples of Composite Objects that you create using the Agfa/Philips DVT. The number of evaluations you run depends on the types of images that you produce. We will not list specific requirements, but ask you to apply good judgment. For example, an Evidence Creator that produces GSPS objects and Encapsulated PDF objecs would evaluate samples from both of those image types.

You need to evaluate and provide the output for at least one DICOM Composite Object. Evaluating other objects can only help your implementation.

References

Instructions

  1. Produce a DICOM Part 10 file containing your composite object for evaluation.
  2. Run the evaluation script for test 283 as follows:
    perl 283/eval_283.pl <output level> <path to file>
  3. The output is stored in 283/grade_283.txt. Submit the file to the Project Manager for evaluation.

Evaluation

Supplemental Information

Test 283 assumes you have installed the DVT on a Windows system. You may elect not to run this test, but you will be required to run test 284. If you choose not to run this test, submit the results for test 284 for this test. Yes, submit 284 for both 283 and 284.

284: DICOM Composite Object Evaluation – DICOM3TOOLS

In this test, you evaluate samples of Composite Objects that you create using the Dave Clunie DICOM3TOOLS. The number of evaluations you run depends on the types of images that you produce. We will not list specific requirements, but ask you to apply good judgment. For example, an Evidence Creator that produces GSPS objects and Encapsulated PDF objecs would evaluate samples from both of those image types.

You need to evaluate and provide the output for at least one DICOM Composite Object. Evaluating other objects can only help your implementation.

References

Instructions

  1. Produce a DICOM Part 10 file containing your composite object for evaluation.
  2. Run the evaluation script for test 284 as follows:
    perl 284/eval_284.pl <output level> <path to file>
  3. The output is stored in 284/grade_284.txt. Submit the file to the Project Manager for evaluation.

Evaluation

Supplemental Information

Test 284 assumes you have installed the MESA tools on a Unix system. You may elect not to run this test, but you will be required to run test 283. If you choose not to run this test, submit the results of test 283 for this test. Yes, you submit 283 for both 283 and 284.

301: Evidence Creator Storage and Commitment

In test 301, the Evidence Creator C-Stores one or more Evidence Documents to the MESA Image Manager. The Evidence Creator also request storage commitment for those documents. The purpose of the test is to check the C-Store command/data set (presumed to be easy) and to check the request for Storage Commitment.

References

Instructions

Run these commands/steps from a terminal emulator (Unix) or DOS/Command Window (Windows).

  1. Set the current directory to $MESA_TARGET/mesa_tests/rad/actors/evdcrt.
  2. Start the MESA servers as described in Starting the MESA Servers above.
  3. Clear the MESA Image Manager of existing data in the database:
    perl scripts/reset_servers.pl
  4. Transmit (C-Store) one or more Evidence Documents to the MESA Image Manager.
  5. Request Storage Commitment of the Image Manager for your Evidence Documents.

If you need to rerun the test, make sure you clear the MESA Image Manager using step 3.

Evaluation

  1. Run this evaluation script:
      perl 301/eval_301.pl <output  level> <AE Title Storage Commit SCU> 
    You have successfully completed when the logfile (301/grade_301.txt) indicates 0 errors.
  2. Submit the logfile 301/grade_301.txt to the Project Manager.

Supplemental Information

The MESA Image Manager does not automatically send Storage Commitment N-Event reports. If your application would like to receive the appropriate Storage Commitment N-Event report, run this script.

Enter script information.

311: Create/Render Encapsulated PDF

In test 311, the Evidence Creator C-Stores one Encapsulated PDFEvidence Documents to the MESA Image Manager. The purpose of the test is to check the C-Store command of the Encapsulated PDF document and to extract the PDF from the DICOM oject.

References

Instructions

Run these commands/steps from a terminal emulator (Unix) or DOS/Command Window (Windows).

  1. Set the current directory to $MESA_TARGET/mesa_tests/rad/actors/imgcrt.
  2. Start the MESA servers as described in Starting the MESA Servers above.
  3. Clear the MESA Image Manager of existing data in the database:
    perl scripts/reset_servers.pl
  4. Transmit (C-Store) one Encapsulated PDF Evidence Documents to the MESA Image Manager.

If you need to rerun the test, make sure you clear the MESA Image Manager using step 3.

Evaluation

  1. Run this evaluation script:
      perl 311/eval_311.pl <output  level> 
    You have successfully completed when the logfile (311/grade_311.txt) indicates 0 errors and the file 311/311.pdf contains the PDF you encapsulated in your document
  2. Submit the logfile 311/grade_311.txt and the file 311/311.pdf to the Project Manager.

Supplemental Information

Evidence Creator Test 500: Display Calibration

Evidence Creators supporting the Consistent Presentation of Images Integration Profile must calibrate their displays in accordance with DICOM PS 3.14. Instructions for this test are included in the document: Display Consistency Test Plan for Image Creator.

Evidence Creator Test 511: Key Image Note 511

In this test, the Evidence Creator will create a Key Image Note that refers to a single image from a series.

References

Instructions

  1. Create/modify the SQL script to identify the Evidence Creator under test.
  2. Start the MESA servers as described in Starting the MESA Servers above.
  3. The steps below are run from the directory $MESA_TARGET/mesa_tests/rad/actors/imgcrt.
  4. Load the data sets into the MESA Image Manager.
  5. perl 5xx/load_img_mgr.pl
  6. Retrieve the study for the patient CRTHREE^PAUL.
  7. Create a Key Image Note with the parameters described below.  Send the DICOM composite object to the MESA Image Manager.

Evaluation

  1. Evaluate the contents of your Key Image Note as follows:
    perl 511/eval_511.pl [-v]

Supplemental Information

If you need to send the note a second time, you should clear the MESA Image Manager first. This will allow the evaluation software to examine your latest object.
 perl  scripts/reset_servers.pl

Template Identifier

2010:DCMR

Required

Document Title

113000:DCM:Of Interest

Required

HAS CONC MOD

CODE 121049:DCM:Language of Content Item and Descendants =
ISO369_2:eng:English

Optional

HAS OBS CONTEXT

CODE 121005:DCM:Observer Type = 121006:DCM:Person

Required

HAS OBS CONTEXT

PNAME 121008:DCM:Person Observer Name = MOORE^STEVE

Required

CONTAINS TEXT

113012:DCM:Key Object Description = Key Object Test 511

Required

Image Reference

Select the image with Image Number 16

Required


Evidence Creator Test 512: Key Image Note 512

In this test, Image Displays will create a Key Image Note that refers to two images from one series.

References

Instructions

  1. Create/modify the SQL script to identify the Evidence Creator under test.
  2. Start the MESA servers as described in Starting the MESA Servers above.
  3. The steps below are run from the directory $MESA_TARGET/mesa_tests/rad/actors/imgcrt.
  4. Load the data sets into the MESA Image Manager.
  5. perl 5xx/load_img_mgr.pl
  6. Retrieve the study for the patient CTFIVE^JIM.
  7. Create a Key Image Note with the parameters described below.  Send the DICOM composite object to the MESA Image Manager.

Evaluation

  1. Evaluate the contents of your Key Image Note as follows:
    perl 512/eval_512.pl [-v]

Supplemental Information

If you need to send the note a second time, you should clear the MESA Image Manager first. This will allow the evaluation software to examine your latest object.
  perl  scripts/reset_servers.pl

Template Identifier

2010:DCMR

Required

Document Title

113007:DCM:For Patient

Required

HAS CONC MOD

CODE 121049:DCM:Language of Content Item and Descendants =
ISO369_2:eng:English

Optional

HAS OBS CONTEXT

CODE 121005:DCM:Observer Type = 121006:DCM:Person

Required

HAS OBS CONTEXT

PNAME 121008:DCM:Person Observer Name = MOORE^STEVE

Required

CONTAINS TEXT

113012:DCM:Key Object Description = Key Object Test 512

Required

Image Reference

Select images with Image Number 67 and 68

Required

Evidence Creator Test 513: Key Image Note 513

In this test, Evidence Creators will create a Key Image Note that refers to two images; each from a different series.

References

Instructions

  1. Create/modify the SQL script to identify the Evidence Creator under test.
  2. Start the MESA servers as described in Starting the MESA Servers above.
  3. The steps below are run from the directory $MESA_TARGET/mesa_tests/rad/actors/imgcrt.
  4. Load the data sets into the MESA Image Manager.
  5. perl 5xx/load_img_mgr.pl
  6. Retrieve the study for the patient MRTHREE^STEVE.
  7. Create a Key Image Note with the parameters described below.  Send the DICOM composite object to the MESA Image Manager.

Evaluation

  1. Evaluate the contents of your Key Image Note as follows:
    perl 513/eval_513.pl [-v]
  2. Submit the output of the results to the Technical Project Manager.
If you need to send the note a second time, you should clear the MESA Image Manager first. This will allow the evaluation software to examine your latest object.
 perl  scripts/reset_servers.pl

Supplemental Information

Table of information to use when creating Key Object Note

Template Identifier

2010:DCMR

Required

Document Title

113004:DCM:For Teaching

Required

HAS CONC MOD

CODE 121049:DCM:Language of Content Item and Descendants = ISO369_2:eng:English

Optional

HAS OBS CONTEXT

CODE 121005:DCM:Observer Type = 121006:DCM:Person

Required

HAS OBS CONTEXT

PNAME 121008:DCM:Person Observer Name = MOORE^STEVE

Required

CONTAINS TEXT

113012:DCM:Key Object Description = Key Object Test 513

Required

Image Reference

Select Image 9 from Series 103
Select Image 19 from Series 104

Required


Evidence Creator Test 521: Consistent Presentation of Images

This test is for Evidence Creators that support the Consistent Display of Images integration profile. Instructions for this test are found in the document Display Consistency Test Plan for Image Creator.

Evidence Creator Test 551: Example Image/GSPS/BSPS and Spatial Registration Objects

The goal of this “test” is to provide samples for other vendors to display.  You should send a “representative sample” of the data produced by your system.

The Scheduled Workflow integration profile only uses images and not presentation state objects. Therefore, Evidence Creator actors supporting the Scheduled Workflow integration profile should submit sample images.

GSPS objects are discussed/supported in the Consistent Presentation of Images integration profile. Evidence Creator actors create GSPS objects (requirement) and may optionally produce images. Evidence Creator actors that support the CPI integration profile should submit the GSPS objects they produce and any images produced. If you use images that are not part of the MESA test set as the basis for your GSPS objects, you should submit those, even if you did not produce them. That will allow other actors to display the original images and the appropriate GSPS objects.

Blended Softcopy Presentation State (BSPS) and Spatial Registration objects are used in the Image Fusion Profile.  Both Evidence Creator and Acquisition Modality actors may create and store BSPS and Spatial Registration objects.  In addition to BSPS objects, you should also submit the datasets (images, Spatial Registrations) you reference within your BSPS object, even if you did not produce them. That will allow other actors to display the original images with the appropriate BSPS and Spatial Registration objects.

Each system should send samples of the Image and/or GSPS, BSPS and Spatial Registration objects that they create to the Image Manager.  If you create BSPS and/or Spatial Registration objects in the Image Fusion profile, you should also send along the image series the BSPS and Spatial Registration objects reference.

These are to be submitted two weeks in advance of the general date for test results to allow other vendors the opportunity to test with them. 

References

Instructions

Either create DICOM Part 10 files and submit to the Project Manager or follow the instructions below.

  1. Start the MESA servers as described in Starting MESA Servers.
  2. Clear the MESA Image Manager (if necessary). From $MESA_TARGET/mesa_tests/rad/actors/evdcrt:
  3. perl scripts/clear_img_mgr.pl
  4. Send sample images/GSPS/BSPS/Spatial Registration objects to the MESA Image Manager.
  5. Tar or zip the files in $MESA_STORAGE/imgmgr/instances
  6. If the file is small enough, you may upload it into the Connectathon webtool as results for this test. If the file is too large, send an email to the Project Manager stating that you are ready to submit results. Do not email the zipped objects. You will receive a response containing instructions for submitting your objects to an ftp site or sending in a CD.

Evaluation

The evaluation of this test comes in the form of feedback from other users of your data. If other users identify issues with your data, you will be asked to work with those users (and Project Manager) to resolve those issues.

Supplemental Information


Evidence Creator Test 552: Example Key Image Note

As with test 551, the goal of this test is to send representative samples to the Project Manager for distribution to other vendors. These samples will be based on tests 511, 512, and 513.

References

Instructions

Either create DICOM Part 10 files with your original DICOM files and submit to the Project Manager or follow the instructions below.

  1. After you complete tests 511, 512, and 513, locate the Key Image notes stored on the MESA Image Manager.  These will be in $MESA_STORAGE/imgmgr/instances.  The first directory level is the Study Instance UID.  You should recognize your Key Image Notes by the Series Instance UID used to identify the next directory.
  2. Tar or zip the files holding your Key Image Notes.
  3. If the file is small enough, you may upload it into the Connectathon webtool as results for this test. If the file is too large, send an email to the Project Manager stating that you are ready to submit results. Do not email the zipped objects. You will receive a response containing instructions for submitting your objects to an ftp site or sending in a CD.

Evaluation

The evaluation of this test comes in the form of feedback from other users of your data. If other users identify issues with your data, you will be asked to work with those users (and Project Manager) to resolve those issues.

Supplemental Information

Evidence Creator Test 1412: PWF CT

Test 1412 is a test of the steps for Post Processing Workflow in a CT 3D reconstruction scenario. CT images will be created and a 3D reconstruction workitem will be scheduled. As an Evidence Creator, you will be asked to query for the post processing worklist and to claim the scheduled workitem. Subsequent post processing workflow steps are not yet implemented.

References

Instructions

  1. Run the active manager test script:
  2. perl 1412/1412_imgcrt.pl
  3. Follow the test instructions

Evaluation

  1. Run the evaluation script:
    perl 1412/eval_1412.pl
    The evaluation script should yield 0 errors for a successful test.
  2. Submit the output of the grade file found in 1412/grade_1412.txt to the Project Manager.

Supplemental Information

Evidence Creator Test 1700: Evidence Document Description

In the Evidence Documents profile, Evidence Documents are defined as DICOM SR objects that are to be used to assist in diagnosis. An example would be measurements on an Ultrasound device.

The purpose of this test is to make sure that Evidence Creators in the Evidence Documents profile understand the content they are to produce is contained in DICOM SR objects according to the Evidence Document Profile. As mentioned above, the most obvious example are Ultrasound measurements. Another example could be a Mammography CAD file. Evidence Documents (in the Evidence Document Profile) are not images, nor are they DICOM SR Diagnostic Reports.

The instructions below are not a joke. We have had experience with this profile indicating users do not understand the intent of the Evidence Documents profile.

References

Instructions

  1. Create a text file and answer the questions below:
    1. What DICOM SOP class is used by your system to generate Evidence Documents? This should be a Structured Report Class.
    2. Describe in 100-500 words why your documents are to be considered evidence and are not merely diagnostic reports or other SR objects.
    3. Describe in 100-500 words what problems other vendors will have in rendering your document or incorporating your results in a diagnostic report.
  2. Name the text file using the convention: CompanyName_1700.txt
  3. Submit the text file to the Project Manager for evaluation.

Evaluation

Project Manager will read the answers provided to make sure your system is operating in the proper context.

Supplemental Information

Evidence Creator Test 1701: Evidence Document Management in Scheduled Workflow

Test 1701 covers evidence document management in scheduled workflow, part of the Evidence Document profile. The test itself is an implementation of the scheme shown in Figure 14.2-1 (see IHE TF Vol I, section 14).

References

Instructions

  1. To run this test, be sure you are in the $MESA_TARGET/mesa_tests/rad/actors/evdcrt directory.  Then, execute:
  2.  perl  scripts/imgmgr_swf.pl 1701 

Evaluation

  1. To evaluate this test:
     perl  1701/eval_1701.pl
    The evaluation script should yield 0 errors for a scuccessful test.
  2. Submit the grade file found in 1701/grade_1701.txt to the Project Manager.

Supplemental Information

Test 283: DICOM Composite Object Evaluation – DVT

In this test, you evaluate samples of DICOM Composite Objects that you create using the Agfa/Philips DVT. The number of evaluations you run depends on the types of DICOM objects that you produce. We will not list specific requirements, but ask you to apply good judgment.
You need to evaluate and provide the output for all the DICOM Composite Object types your system produces. Evaluating other objects can only help your implementation.

References

Instructions

  1. Produce a DICOM Part 10 file containing your composite object for evaluation.
  2. Run the evaluation script for test 283 as follows:
    perl  283/eval_283.pl <output level> <path to file>
  3. The output is stored in 283/grade_283.txt. Submit the file to the Project Manager for evaluation.

Evaluation

Supplemental Information

Test 283 assumes you have installed the DVT on a Windows system.

 

Nuclear Medicine Specific Tests

2800: Result Screen Export Documentation

Evidence Creator test 2800 is a documentation test where the developer documents how the Evidence Creator satisfies the requirements listed in Rad TF-2: 4.18.4.1.2.4

Reference:

Rad TF-2: 4.18.4.1.2.4

Instructions

  1. Read Rad TF-2: 4.18.4.1.2.4
  2. Create a text file labeled: SYSTEM_2800.txt
  3. Copy and answer these questions in the text file and submit to the Project Manager:
    1. Does your system produce Dyamic Result screens? Are these stored as Multi-Frame Secondary Capture (MFSC) IODs? If not, then you are not following the specifications properly.
    2. Does your system produce Static Result screens?
    3. Does your system produce Greyscale Result screens?
    4. Does your system produce Color Result screens?
    5. Does your system produce Multi-Frame Secondary Capture (MFSC) IODs for storing Static Result Screens?
    6. Does your system produce DICOM Secondary Capture (SC) IODs for storing Static Result Screens?
    7. Does your system present color results? If so, does your system export those results in 24-bit RGB? (If not, then you are not following the specifications properly).
    8. List the value stored in Conversion Type (0008 0064)
    9. List the value stored in Series Description (0008 103E)
    10. List the value stored in Derivation Descripton (0008 2111)

Evaluation

Supplemental Information

Evidence Creator Test 2801: NM Reconstructed Images Special Requirements

Reference:

Rad TF-2: 4.18.4.1.2.3

Instructions

In test 2801, reconstructed tomographic datasets are tested for these attributes:

0054 0022 Detector Information Sequence
>> 0020 0037 Image Orientation
0018 0088 Spacing Between Slices

Evaluation

Supplemental Information

Evidence Creator Test 2802: NM Reconstructed Images Cardiac Views

Reference:

Rad TF-2: 4.18.4.1.2.3

Instructions

In test 2802, reconstructed tomographic datasets are tested for these attributes:

0054 0220 View Code Sequence
0054 0500 Slice Progression Direction
0040 0555 Acquisition Context Sequence

Evaluation

Supplemental Information

Evidence Creator Test 2803: NM Result Export

Reference:

Rad TF-2: 4.18.4.1.2.4

Instructions

Evaluation

Supplemental Information


2804: NM Result Export Screen 1

Test 2804 examines Export Result Screen data for values as defind in Rad TF-2: 4.18.4.1.2.4

Reference

Rad TF-2: 4.18.4.1.2.4

Instructions

  1. Produce one or more NM Result Export Screens per Rad TF-2: 4.18.4.1.2.4.
  2. Create a DICOM Part 10 file or store in the MIR CTN format (IVRLE, no preamble, no group 0002).
  3. Copy/ftp/carrier pigeon the file (or files) to the MESA box/computer.

Evaluation

  1. Run the evaluation script for test 2804:
  2. perl 2804/eval_28-4.pl <log level> FILE1 
  3. Submit the evaluation log (2804/grade_2804.txt) to the Project Manager.

Supplemental Information

Log level is a value from 1 to 4 (1 is low, 4 is more messages). When sending the evaluation output to the Project Manager, use a value of 3 or 4.

Cardiology Specific Tests

 

Evidence Creator Test 20640: Evidence Creation – Cath –Vendor Interoperability

Test 20640 tests the creation and content of an SR with a Cath template. The ED Profile assumes that the Evidence Creator is part of Cath Scheduled Workflow. Although the acquisition of a Cath Image is not part of the Evidence Documents Profile, the transactions are included here as test set up, but not explicity tested. See the Radiology Technical Framework Volume 1:14 for a more complete explanation.


The purpose of this test is to collect SR object/cath templates from all Evidence Creator actors prior to the Connectathon. These vendors/actors are required to submit SR objects for every Cath template and SR SOP Class supported. These files should be submitted to the IHE web tool as part of the results of these tests. These files will be used by the Image Display vendors/actors as Test Case 20650. It is requested that this test, in particular, be completed at least two weeks in advance of the MESA test completion date to allow the Image Display actors to test the display of each of these objects and to allow time for communication if there is a problem.

Instructions

There is no MESA software required for this test.   The vendor should create a file using the naming convention of: CompanyName_Product_ 20640_EC_n_2007.doc , where n is any number to that you make up to differentiate the files if the SR Vendor has submitted multiple objects. 

If the file is small enough, you may upload it into the Connectathon webtool as results for this test. If the file is too large, send an email to the Project Manager stating that you are ready to submit results. Do not email the zipped objects. You will receive a response containing instructions for submitting your objects to an ftp site or sending in a CD.

Evaluation

The evaluation of this test comes in the form of feedback from vendors who try to display the contents of your objects. If they find issues in displaying the studies, you will be asked to work with those vendors (and the Project Manager) to resolve those issues

Supplemental Information


Evidence Creator Test 20641: Evidence Creation – Echo –Vendor Interoperability

Test 20641 tests the creation and content of an SR with an Echo template. The ED Profile assumes that the Evidence Creator is part of Echo Scheduled Workflow. Although the acquisition of an echo image is not part of the Evidence Documents Profile, the transaction is included here as test set up, but not explicity tested. See the Radiology Technical Framework Volume 1:14 for a more complete explanation.


The purpose of this test is to collect SR object/cath templates from all Evidence Creator actors prior to the Connectathon. These vendors/actors are required to submit SR objects for every Echo template and SR SOP Class supported. These files should be submitted to the IHE web tool as part of the results of these tests. These files will be used by the Image Display vendors/actors as Test Case 20651. It is requested that this test, in particular, be completed at least two weeks in advance of the MESA test completion date to allow the Image Display actors to test the display of each of these objects and to allow time for communication if there is a problem.

Instructions

There is no MESA software required for this test.   The vendor should create a file using the naming convention of: CompanyName_Product_ 20641_EC_n_2007.doc , where n is any number to that you make up to differentiate the files if the SR Vendor has submitted multiple objects. 

If the file is small enough, you may upload it into the Connectathon webtool as results for this test. If the file is too large, send an email to the Project Manager stating that you are ready to submit results. Do not email the zipped objects. You will receive a response containing instructions for submitting your objects to an ftp site or sending in a CD.

Evaluation

The evaluation of this test comes in the form of feedback from vendors who try to display the contents of your objects. If they find issues in displaying the studies, you will be asked to work with those vendors (and the Project Manager) to resolve those issues

Supplemental Information

 

Evidence Creator Test 20700:  SOP Class Support

Instructions

There are no test steps to execute for this test.  Instead, create a text file which lists all of the SOP classes which your Evidence Creator is capable of creating in the STRESS profile.  Your file should have the following naming convention:    CompanyName_Product_20700_EC_2006.

Submit the text file to the Technical Project Manager as the results for this test.

 


Evidence Creator Test 20740: Stress Profile - Sample Datasets – Vendor Interoperability

The purpose of this test is to collect sample DICOM objects from all Evidence Creator actors in the STRESS profile prior to the Connectathon. You should submit samples for every SOP Class supported. These files should be submitted to the IHE web tool as part of the results of these tests. These files will be used by the Image Display vendors/actors. It is requested that this test, in particular, be completed at least two weeks in advance of the MESA test completion date to allow the Image Display actors to test the display of each of these objects and to allow time for communication if there is a problem.

Instructions

MESA: There is no MESA software required for this test.
The vendor should create a file using the naming convention of: CompanyName_Product_ 20740_EC_n_2006.<ext> , where n is any number to that you make up to differentiate the files if you submit multiple objects.

If the file is small enough, you may upload it into the Connectathon webtool as results for this test. If the file is too large, send an email to the Project Manager stating that you are ready to submit results. Do not email the zipped objects. You will receive a response containing instructions for submitting your objects to an ftp site or sending in a CD.

Evaluation

The evaluation of this test comes in the form of feedback from vendors who try to display the contents of your objects. If they find issues in displaying the studies, you will be asked to work with those vendors (and the Project Manager) to resolve those issues

Supplemental Information

 

Image Fusion (FUS) Profile Tests

Evidence Creators in the Image Fusion Profile must execute the tests in this section. The test images that your Evidence Creator will use are provided in the MESA_STORAGE distribution in the FUSION directory..

Evidence Creator Test 3510: Create and Store Spatial Registration – same FoR

<This test is not yet implemented in the MESA tools.>

In this test, the Evidence Creator will create a Spatial Registration Object that spatially aligns two series of images.  These series have the same Frame of Reference.  This tests transaction [RAD-56] in the Image Fusion Profile and Storage Commitment [RAD-10].

References

Rad TF-1: 20.4.2
Rad TF-3: 4.56
DICOM PS3.3 – 2004 A.30

Instructions

  1. Load the test images for 3510 onto your Evidence Creator.  There are two series of images for patient Chestnut^C.
  2. Use your application to register the two datasets (registration could be based on fiducials or image content).  C-STORE the DICOM Spatial Registration Object to the MESA Image Manager.
  3. Request Storage Commitment.

Evaluation

  1. Evaluate the contents of your Spatial Registration IOD and Storate Commitment N-ACTION as follows:
  2. perl 3510/eval_3510.pl  <log_level>

If you need to send the Spatial Registration object a second time, you should clear the MESA Image Manager first.  This will allow the evaluation software to examine your latest object.

     perl  scripts/reset_servers.pl

Supplemental Information

Evidence Creator Test 3512: Create and Store Spatial Registration – different FoR

In this test, the Evidence Creator will create a Spatial Registration Object that spatially aligns two series of images received from the MESA image server.  These series have different Frames of Reference.  The image dataset can be copied to the Evidence Creator from the CD/DVD provided with the MESA tools. Alternatively, it can be C-STOREd using the test script (below) for this test

This tests transaction [RAD-56] in the Image Fusion Profile and Storage Commitment.

Note: The Registration created in this test will be used again in subsequent tests 3514 and 3516.

References 

Rad TF-1: 20.4.2
Rad TF-3: 4.56
DICOM PS3.3 – 2004 A.30

Test description/steps for test 3512.

Instructions

 Create/modify the SQL script to identify the Evidence Creator under test.

  1. If you are not using the test script to C-STORE the images to your Evidence Creator, load the test images for 3512 onto your Evidence Creator.  There are two series of images for patient ZZ^Z.
  2. Make sure the MESA servers have been started as described in the Starting MESA Servers section above.
  3. From the $MESA_TARGET/mesa_tests/rad/actors/evdcrt directory, run the following script:
  4. perl scripts/evdcrt_fusion.pl 3512 <log_level>

Evaluation

  1. Evaluate the contents of your Spatial Registration IOD as follows:
     perl 3512/eval_3512.pl <log_level> <Storage Commit AE Title>

If you need to send the Spatial Registration object a second time, you should clear the MESA Image Manager first.  This will allow the evaluation software to examine your latest object.

      perl scripts/reset_servers.pl

Supplemental Information

Evidence Creator Test 3514: Create BSPS with Spatial Registration

In this test, the Evidence Creator (or Modality) creates a valid DICOM BSPS object which references a Spatial Registration. You may reference the Spatial Registration Object created in test 3512 or create a new one.

You must store both the BSPS and Spatial Registration objects during this test. You can C-STORE the two objects in any order.

Storage Commitment may be sent, but it will not be evaluated in this test.

References

Rad TF-3: 4.57.4.1.2

Test description/steps for test 3514.

Instructions

  1. You should already have the test images and Spatial Registration object from test 3512 on your Evidence Creator. If not, the images can be C-STOREd to your Evidence Creator within the test script.
  2. Make sure the MESA servers have been started as described in the Starting MESA Servers section above.
  3. From the $MESA_TARGET/mesa_tests/rad/actors/evdcrt directory, run the following script:
  4. perl scripts/evdcrt_fusion.pl 3514 <log_level>

Evaluation

  1. Evaluate the contents of your BSPS and Spatial Registration object as follows:
     perl 3514/eval_3514.pl <log_level> <EC C-STORE AE Title>

If you need to send the BSPS and Spatial Registration objects a second time, you should clear the MESA Image Manager first.  This will allow the evaluation software to examine your latest objects.

  perl  scripts/reset_servers.pl

Supplemental Information

This test uses the same source PT/CT images from 3512. You might choose to re-use the SPATIAL object you created for 3512.

Evidence Creator Test 3516: Modify Existing Spatial Registration

<This test is not yet implemented in the MESA tools.>

In this test, the Evidence Creator will modify Spatial Registration from test 3512.  The Evidence Creator will modify the transformation for one or both datasets and create a new Spatial Registration instance.

Note: Test 3512 must be run before this one.

References

Rad TF-3: 4.56.4.1.12

Instructions

  1. The Evidence Creator should locate the Spatial Registration created for test 3512.
  2. Use your application to modify the transformation of the dataset(s)
    - perform a horizontal flip
    - rotate 90 degrees right
  3. Send the new DICOM Spatial Registration object to the MESA Image Manager.

Evaluation

  1. Evaluate the contents of your Spatial Registration IOD as follows:
     perl 3516/eval_3516.pl [-v]

If you need to send the Spatial Registration object a second time, you should clear the MESA Image Manager first.  This will allow the evaluation software to examine your latest object.

     perl  scripts/reset_servers.pl

Supplemental Information

Evidence Creator Test 3540: Create and Store BSPS IOD – same FoR

In this test, the Evidence Creator will create a Blended Softcopy Presentation State (BSPS) object that specifies how to blend for display the two series of images.  These series have the same Frame of Reference.  This tests transaction [RAD-57] in the Image Fusion Profile and Storage Commitment [RAD-10]. Spatial Registration is tested in another test.

Note:  The BSPS created in this test will be used again in the next test, 3541. 

References

Rad TF-1: 20.4.3
Rad TF-3: 4.57
DICOM PS3.3 – 2004 A.30

Test description/steps for test 3540.

Instructions

  1. Load the test images for 3540 onto your Evidence Creator.  There are two series of images for patient Papaya^P. You can also use the test script below to C-STORE these images to your Evidence Creator.
  2. Make sure the MESA servers have been started as described in the Starting MESA Servers section above.
  3. From the $MESA_TARGET/mesa_tests/rad/actors/evdcrt directory, run the following script:
  4. perl scripts/evdcrt_fusion.pl 3540 <log_level>

Evaluation

  1. Evaluate the contents of your BSPS IOD as follows:
     perl 3540/eval_3540.pl <log_level>  <Storage Commit AE Title>

If you need to send the BSPS object a second time, you should clear the MESA Image Manager first.  This will allow the evaluation software to examine your latest object.

     perl  scripts/reset_servers.pl

Supplemental Information

Evidence Creator Test 3541: Modify Existing BSPS

In this test, the Evidence Creator will modify the Blended Softcopy Presentation State (BSPS) object created in test 3540.  Test 3540 must be run before this one.  The Evidence Creator will modify the transparency of the SUPERIMPOSED dataset and create a new BSPS instance and store a new BSPS object, with Storage Commitment, to the MESA Image Manager.

References

Rad TF-3: 4.57.4.1.2

Test description/steps for test 3541.

Instructions

  1. The Evidence Creator should locate the Blended Softcopy Presentation State created for test 3540.
  2. Make sure the MESA servers have been started as described in the Starting MESA Servers section above.
  3. From the $MESA_TARGET/mesa_tests/rad/actors/evdcrt directory, run the following script:
  4. perl scripts/evdcrt_fusion.pl 3541 <log_level>

Evaluation

  1. Evaluate the contents of your BSPS IOD as follows:
     perl 3541/eval_3541.pl <log_level>  <Storage Commit AE Title>

If you need to send the BSPS object a second time, you should clear the MESA Image Manager first.  This will allow the evaluation software to examine your latest object.

     perl  scripts/reset_servers.pl

Supplemental Information

 

Mammography Specific Tests

Evidence Creator Test 3916: Sample Mammo CAD SR Objects - Vendor Interoperability

In this “test”, you use the MESA tools to submit a sample DICOM Mammography CAD SR and the associated imagee objects. The purpose is to identify interoperability problems before the Connectathon by distributing these object to Image Display actors for them to render.

  • You should send at least one example
  • You must also send any 'For Presentation' images that are referenced by the Mammography CAD SR instance.
  • Finallly, you must send any 'For Processing' SOP instances referenced in the Source Image Sequence of the 'For Presentation' images.
  • Note that your DICOM objects MUST NOT contain real patient demographics. 
  • Note that your SR(s) will be distributed to other vendors.

If possible, encode as many attributes identified in RAD TF-2:4.16.4.2.2.1.1.8 to enable Image Displays to be thoroughly exercised.

References

Instructions

    1. Start the MESA servers as described in Starting MESA Servers.
    2. Clear the MESA Image Manager.
      perl  scripts/clear_img_mgr.pl
    3. C-STORE DICOM objects to the MESA Image Manager.
    4. Locate the DICOM objects stored by the MESA Image Manager.  These are in $MESA_STORAGE/imgmgr/instances.  Tar or zip these files
    5. If the file is small enough, you may upload it into the Connectathon webtool as results for this test. If the file is too large, send an email to the Project Manager stating that you are ready to submit results. Do not email the zipped objects. You will receive a response containing instructions for submitting your objects to an ftp site or sending in a CD.
    6. You may submit more than one set.

Evaluation

The evaluation of this test comes in the form of feedback from vendors who try to display the contents of your objects. If they find issues in displaying the studies, you will be asked to work with those vendors (and the Project Manager) to resolve those issues

Supplemental Information