There are 800,000 shipments of hazardous material (HM) each day in the United States. Motor carrier HM shipments are potential targets for terrorist activities. A subset of these daily shipments poses a greater risk of harm to the public because of the type of material being transported. These include explosives, poisonous gas, and radioactive material. A means must be developed to ensure the safety and security of these specific HM shipments from origin to destination. Security measures need to be taken in each stage of the shipment-pick up, delivery, receipt-and for each element of the shipment-driver, vehicle, and cargo. Additional measures must also be in place to handle emergency situations. It is a joint responsibility of HM shippers and carriers, the FMCSA, and State safety agencies to ensure the safe transportation of these materials.

The objective of the HM Transportation Safety and Security Operational Test is to quantify the security costs and benefits of an operational concept that applies technology and improved enforcement procedures to HM Transportation. A field operational test shall be conducted to demonstrate an approach that ensures the safety and security of HM shipments from origin to destination.

The evaluation team will evaluate whether or not the HM Transportation application met it's objective of ensuring the safety and security of the specific HM shipments from origin to destination.

Hazardous Material (HM) Transportation Safety and Security Operational Test.

This statement of work begins a new task and is intended to verify that the proposed system specifications and design meet the functional requirements and address, at a minimum, the following risk areas: driver verification, off-route vehicle alerts, stolen vehicles (both tractors and trailers), unauthorized drivers, cargo tampering, and suspicious cargo deliveries. Suspicious cargo deliveries include the shipment of HM to facilities that would not normally use the HM in their business operations and the shipment of different HM that when combined could pose a security risk.

The scope of the project shall be organized in three stages: the pick-up of HM from shipper, the en-route transportation of the HM, and the delivery to the receiver at the final destination.

The evaluation activities for the HM Transportation Safety and Security Operational Test shall include:

• Coordination with project partners to ensure that evaluation requirements can be accommodated
• High resolution photographic documentation of project, especially early context setting and then major new system capabilities
• The development of an Evaluation Strategy/Plan
• The development of a Detailed Test Plan
• Data Collection prior to system deployment
• Statistical Analysis of data collected prior to system deployment
• Data Collection post system deployment
• Statistical Analysis of data collected post system deployment
• Evaluate the risk/threat assessment presented by the contractor under Task 1 of the operational test
• And the necessary reporting and managing of the evaluation throughout the completion of this Phase.

This evaluation is intended to serve as a System Impact Study. System Impact Studies are intended to provide quantitative system impact findings. This evaluation should provide good quality system impact data in such areas as safety performance, system operational performance, and customer satisfaction, along with qualitative documentation as to how the project was completed successfully.

This evaluation should examine how well the HM Transportation Safety and Security Operational Test meets the objectives. This evaluation shall be accomplished through a combination of evaluation studies that examine system component performance.

The evaluation contractor will be required to submit an interim report at the end of Phase I that will provide the following:

- Evaluation strategy
- Updated project and evaluation schedule
- Test plans
- "Before" system performance data
- Statistical analysis of "Before" system performance data along with a description of how it will be compared to "After" system performance data
- Identify potential issues and/or challenges

The purpose of the interim report is to allow U.S. DOT program managers to determine if the HM Transportation Safety and Security project is progressing as planned or if changes in the work statement or priorities are necessary. Upon review of the Phase I report, the COTR will instruct the evaluation contractor to proceed to Phase II for this System Impact Evaluation Project. Phase II is defined as the completion of a System Impact Evaluation in which "after" system performance data are collected and evaluated in comparison with "before" system performance data.

For System Impact studies, Phase I shall include the following tasks:

- Development of an evaluation plan
- Development of detailed test plan
- Collection of baseline or "Before" system performance data
- Statistical analysis of baseline or "Before" system performance data
- Review and comment on risk/threat assessment produced under Task 1 of the operational test.

An explanation of what is expected from each Phase I task for System Impact studies is provided below.

Evaluation Plan

An evaluation plan shall be delivered for the HM Transportation Safety and Security project. This evaluation plan shall refine the preliminary evaluation approach and expand it to include at a minimum: an introduction; background; objectives of the evaluation; a system description; description of the evaluation management structure; evaluation approach to include identification of specific detailed test plans needed; estimated level of effort; work breakdown structure; evaluation schedule; and, data management and deliverable reports. The evaluation contractor shall modify the evaluation approach based on comments received from the COTR.

Detailed Test Plan

The number and nature of individual test plans will be identified in the evaluation plan. A Detailed Test Plan will be included in the HM Transportation Safety and Security evaluation plan. This plan shall, in general, describe full-scale field tests of the impacts of the project with an ability to compare impacts in a before/after or with/without environment. Multiple individual test plans are expected. An individual test plan could address similar impacts resulting from different components in the project, or it could address multiple impacts resulting from a specific component. Each individual test plan shall address, at a minimum, the following: Individual test plan objective; approach, describing the overall test strategy, data collection methods, the analysis methods, and key supporting conditions needed to conduct the test; Test schedule; Pre-test activities; Test Activities; Post-test activities; Data requirements; Data analysis; Report format and expected contents; and Estimated resources required to complete all activities described in the individual test plans.

Collection of Baseline Data

The Evaluation contractor shall collect sufficient baseline data to support quantitative measurement of effects the project is exerting on the transportation system. The types of data to be collected, data elements, sample sizes, and collection period will be defined in the detailed Test Plans. The evaluation contractor shall coordinate with projects' management team(s) to maximize the use of baseline data collection products available from archived databases.

Statistical Analysis of Baseline or "Before" System Performance Data

The evaluation contractor shall perform statistical analyses on the baseline system performance data to determine such things as, mean, average, variance, maximum, minimum, distribution, etc. of system performance data. In addition, a statistical plan shall be developed to test hypotheses of changes in system performance before and after implementation of the HM Transportation Safety and Security technologies. Given the nature of this operational test, this plan may include a method for assessing changes in the safety and security risks.

Review and Comment on Risk/Threat Assessment

The evaluation contractor shall evaluate and provide comments on the risk/threat assessment produced under Task 1 of the operational test. The risk/threat assessment will serve to frame the safety and security risks being addressed by this operational test and will be used as the basis for developing the Concept of Operations. The assessment will categorize the threats and assist and be the basis for the prioritization of countermeasures.

After approval is granted by the COTR, the evaluation contractor shall continue with all planned aspects of Phase II of the evaluation as specified in the Evaluation Plan and submitted Test Plan. Phase II shall include the following tasks:

- Collection of post or "After" system performance data
- Statistical analysis of post or "After" system performance data
- Assessment of reduced risks given the performance data obtained through the operational test
- Final System Impact Report
- And the necessary reporting and managing of the evaluation throughout the completion of Phase II.

Final System Impact Report

The Final System Impact Report shall include the following findings:

• The safety and security benefits of the proposed operational concept that includes new technologies and procedures
• The costs of the proposed technical approach. The cost shall be broken down on a per vehicle basis. An estimate for the total cost for the HM motor carrier industry to deploy the technical approach shall be calculated.
• Security gaps identified during the test
• Performance specifications for required hardware and software
• Deployment strategy to support widespread deployment in the HM industry. This strategy shall include an implementation plan and operations and maintenance requirements for the proposed technologies

The following dates have been determined for the delivery of reports:

A monthly informal status report shall be submitted to the COTR or designated contractor. The informal status report shall be provided via e-mail on the first business day of each month. The COTR may request informal briefings as necessary to support this evaluation effort.

An Evaluation Plan for the HM Transportation Safety and Security project shall be provided to the COTR within 3 months of task approval, for review and comment (15 copies). Comments will be provided to the evaluation contractor within 3 weeks from the date the evaluation plan was submitted. These comments and guidance shall be incorporated into the final evaluation plan. The evaluation plan shall include an evaluation deliverable schedule. This schedule will include dates for the following, at a minimum:

An interim briefing (presented to the COTR and the HM Working Group). This briefing is to include a report on the progress of the project and the subsequent evaluation along with an initial assessment of the usefulness of this project to provide system impact information.

• Submittal of the Draft Phase I Report for the HM Transportation Safety and Security project.
• Submittal of the Final Draft Phase I Report for the HM Transportation Safety and Security project (including comments received from the COTR and HM Working Group)
• Submittal of the Draft Phase II Report for the HM Transportation Safety and Security project.
• Submittal of the Final Draft Phase II Report for the HM Transportation Safety and Security project (including comments received from the COTR and HM Working Group).

High-resolution photography that sets the context of the project shall be provided within 2 months of task approval. This shall be delivered on CD. Photography documenting significant project milestones shall be provided within 1 month from implementation of each project facet.