OFFICE OF AUDITOR OF ACCOUNTS REQUEST FOR PROPOSAL FOR INDEPENDENT VERIFICATION AND VALIDATION (IV&V) SERVICES JANUARY 8, 2010 DEPARTMENT OF HEALTH AND SOCIAL SERVICES DIVISION OF CHILD SUPPORT ENFORCEMENT SOFTWARE VERIFICATION AND VALIDATION SERVICES FOR THE REPLACEMENT OF THE DELAWARE AUTOMATED CHILD SUPPORT ENFORCEMENT SYSTEM (DACSES) RFP10-CPA-01 by State of Delaware Office of Auditor of Accounts Townsend Building, Suite 1 401 Federal Street Dover, Delaware 19901 OFFICE OF AUDITOR OF ACCOUNTS REQUEST FOR PROPOSAL TABLE OF CONTENTS I. INTRODUCTION A. General Information B. Term of Engagement C. Subcontracting II. NATURE OF SERVICES REQUIRED A. General B. Scope of Work to be Performed C. Standards to be Followed and Definition of Terms D. Deliverables E. Special Considerations F. Contract Documentation Retention and Access to Contract Documentation III. DESCRIPTION OF THE GOVERNMENT A. Location of the Work IV. TIME REQUIREMENTS A. Proposal Calendar B. Date Contract May Commence C. Contract Schedule D. Entrance and Exit Conferences V. ASSISTANCE TO BE PROVIDED TO THE CONTRACTOR AND REPORT PREPARATION A. Assistance and Information to be Provided from the State VI. PROPOSAL REQUIREMENTS A. General Requirements 1. Mandatory Pre-proposal Conference 2. Inquiries 3. Submission of Proposals B. Technical Proposal 1. Required Format 2. Mandatory Criteria 3. Firm Qualifications, Experience, and Requirements 4. Partner, Supervisory, and Staff Qualifications and Experience 5. Prior Engagements with AOA 6. Similar Engagements with Other Government Entities 7. Specific Engagement Approach 8. Identification of Anticipated Potential Problems C. Cost Proposal ` 1. Total All-Inclusive Maximum Cost 2. Rates by Partner, Specialist, Supervisory, and Staff Level Times Hours Anticipated for Each 3. Out-of-Pocket Expenses Included in the Total All-inclusive Maximum Cost and Reimbursement Rates 4. Rates for Additional Professional Services 5. Manner of Payment D. Submission of Proposals E. Modifications to Proposals VII. EVALUATION PROCEDURES A. Evaluation Committee B. Review of Proposals C. Evaluation Criteria 1. Mandatory Elements 2. Technical Qualifications 3. Cost D. Reservation of Rights E. Oral Presentations F. Confidentiality of Documents G. Final Selection H. Right to Reject Proposals VIII. CONTRACT CONDITIONS A. Term B. Preclusion from Resulting Contracts C. Compensation D. Non-Appropriation E. Notice F. Formal Contract and Purchase Order G. Indemnification H. Compliance I. Insurance J. Non-Discrimination K. Covenant Against Contingent Fees L. Contract Documents M. Applicable Law N. Conflict of Interest O. Audits of, Access to, and Retention of Records P. Scope of Agreement APPENDICES A. Process Areas Within the Software Development Life Cycle for Independent Verification and Validation of the DACSES Project B. Schedule of Professional Fees and Expenses C. ACF Prioritization, Probability and Severity Matrix OFFICE OF AUDITOR OF ACCOUNTS REQUEST FOR PROPOSAL I. INTRODUCTION A. General Information In accordance with 29 Del. C. Chapter 69, Subchapter VI, Professional Services, the Office of Auditor of Accounts (AOA) is distributing this Request for Proposal (RFP) to, and requesting proposals from qualified firms to provide Independent Verification and Validation (IV&V) services for the Delaware Department of Health and Social Services, Division of Child Support Enforcement’s (DCSE) replacement of the Delaware Automated Child Support Enforcement System (DACSES) (see Section B., "Scope of Work to be Performed"). The IV&V services are verification and validation activities performed by a vendor that is not under the control of the organization that is developing the new automated system. These services will be for one 12-month base year period followed by four consecutive 12-month option year periods for a total period of performance if all option periods are exercised of 60 months. The selected Contractor will, at a minimum, perform one initial IV&V review at the contract’s commencement, and conduct subsequent IV&V reviews at six-month intervals thereafter. The Contractor and any Subcontractors receiving a contract as a result of this RFP will be restricted from bidding on any other RFP associated with the State’s Department of Health and Social Services, its Division of Child Support Enforcement, and its DACSES Project. This restriction has been established to prevent conflicts of interest within the DACSES Project. In addition, the Contractor and Subcontractor will be prohibited from employing individuals that have previously been employed on the DACSES Project, or were employed on any other project or system design or development activity that provided support to or otherwise interfaced with the DACSES Project. These services are to be performed in accordance with Institute of Electrical and Electronics Engineers (IEEE) Standard 1012-2004; other applicable, lifecycle- appropriate IEEE Standards (e.g., 12207 Software Life Cycle Process; 703 Software QA Plans; 1074 Developing Software Project Lifecycle Process; 828 Configuration Management Plans; and 830 Requirement Specifications, etc., to name a few) in assessing the State’s DACSES Replacement Project. Further, the IV&V Service Provider will employ the Capability Maturity Model Integrated (CMMI), and the Project Management Institute’s Project Management Body of Knowledge (PMBOK) Third Edition, and the PMBOK - Government Extension, as additional standards by which to assess the DACSES Replacement Project. Where an offeror has a similar, corresponding, but different set of minimum standards than those cited above, the offeror will be expected to cross-reference or otherwise map how their own standards meet the same level of detail and scope of review as the industry standards for IV&V cited herein (e.g., CMMI, PMBOK, and IEEE.) Failure to provide this cross-referencing of standards in the offeror’s proposal will be deemed as being non-responsive to this solicitation for purposes of evaluation of the offeror’s proposal. There is no expressed or implied obligation for AOA to reimburse responding firms for any costs or expenses incurred in preparing proposals in response to this request. AOA will not pay any costs or expenses incurred by any firm associated with any aspect of responding to this RFP, including proposal preparation, printing, or delivery, or the negotiation process. Additionally, no indirect reimbursements (e.g., in the form of credits or reductions to any agreed upon compensation) shall be made to any responding firm by AOA for any such costs or expenses. A mandatory pre-proposal conference for all the firms interested in submitting a proposal will be held at 1:00 p.m. on January 21, 2010, at the Division of Child Support Enforcement, 84A Christiana Road, New Castle, DE 19720, in Room 116, to answer questions about this contract engagement. After this mandatory pre-proposal conference, any inquiries concerning the RFP should be addressed to both of the following AOA personnel. * Robert Hicks, CPA, PMP, Deputy Auditor 302-577-5264 (Robert.Hicks@state.de.us) * Candace Casto, CGFM, Senior Manager, Contracts & Administration 302-857-3910 (Candace.Casto@state.de.us) To be considered, five copies of the technical proposal must be sent to Robert Hicks, CPA, PMP, Deputy Auditor, and five copies of the cost proposal must be sent to Candace Casto, CGFM, Senior Manager, Contracts and Administration, both at Office of Auditor of Accounts, Townsend Building, Suite 1, 401 Federal Street, Dover, DE 19901 by 3 p.m. (EST) on February 4, 2010. ALL PROPOSALS MUST BE DELIVERED TO AOA’S THIRD FLOOR OFFICE IN THE TOWNSEND BUILDING, SUITE 1, 401 FEDERAL STREET, DOVER, DE 19901. IF USING FEDERAL EXPRESS OR ANOTHER COMPANY, PLEASE DO NOT DELIVER TO THE DIVISION OF CORPORATIONS IN THE TOWNSEND BUILDING. AOA reserves the right, exercised in its sole discretion, to reject any or all proposals submitted. Proposals submitted may be evaluated by the following: Director of Audit Services, Deputy Auditor, Senior Manager, and Audit Manager. A representative from the offices of DCSE and Federal Office of Child Support Enforcement, State and Tribal Systems (OCSE) may also be on the Evaluation Committee. During the evaluation process, AOA, DCSE, and Federal OCSE’s staff reserves the right, where it may serve the State of Delaware’s best interests, to request additional information or clarifications from proposers, or to allow corrections of errors or omissions. The AOA, DCSE, and Federal OCSE’s staff may also request firms who are submitting proposals to make oral presentations as part of the evaluation process. AOA reserves the right to retain all proposals submitted and to use any ideas in a proposal regardless of whether that proposal is selected. Submission of a proposal indicates acceptance by the firm of the conditions contained in this RFP, unless clearly and specifically noted in the proposal submitted and confirmed in the contract between AOA and the firm selected. Following the notification of the selected firm, a contract will be executed between both parties. B. Term of Engagement These services will be for a 12-month base year period followed by four consecutive 12-month option year periods for a total period of performance if all option periods are exercised of 60 months subject to the satisfactory negotiation of terms (including a cost acceptable to both AOA and the selected firm) and the annual availability of an appropriation by the Legislature. C. Subcontracting The work may be subcontracted. Firms submitting proposals are encouraged to consider subcontracting portions of the engagement to small firms or firms owned and controlled by socially and economically disadvantaged individuals. If this is to be done, that fact, and the name of the proposed subcontracting firms, must be clearly identified in the proposal. Subcontractors, including subcontractor personnel, are subject to the same restrictions on bidding with the DHSS, DCSE, and DACSES Project as is the Contractor for this contract. Further, the AOA reserves the right to reject and require replacement of any individual proposed for or engaged on this contract at any time. The names and resumes of the staff who will be involved in the work must be included in the proposal. Personnel may be changed at the discretion of the proposer provided that replacements have substantially the same or better qualifications or experience. Following the award of this contract, no additional subcontracting will be allowed without the express prior written consent of AOA, DCSE, and OCSE. II. NATURE OF SERVICES REQUIRED A. General AOA is soliciting the services of qualified firms to provide independent verification and validation (IV&V) services for the Delaware Department of Health and Social Services, Division of Child Support Enforcement’s (DCSE) replacement of the Delaware Automated Child Support Enforcement System (DACSES) (see Appendix A for Statement of Work). These services are to be performed in accordance with the provisions contained in this RFP. B. Scope of Work to be Performed Using pre-defined checklists and similar tools founded on industry standards, the IV&V Service Provider staff will interview and observe the DACSES Project Management staff, CSE Program staff, the DACSES Project Development Contractor staff (including any sub-contractors), observe project meetings and activities to understand the processes, procedures, and tools used in the CSE Program and DACSES Project environments, and review and analyze for adherence to accepted, contractually-defined industry standards, all applicable and available documentation. As a result of these interactions and reviews of the applicable DACSES Project documentation, the IV&V Service Provider will produce a structured, exception-based Semi-annual assessment report that objectively illustrates the strengths and weaknesses of the Project. The IV&V Service Provider will also provide recommendations for correcting the weaknesses that the assessment reports identify. To ensure the independence of the IV&V effort, all deliverables will be submitted concurrently to OCSE when a copy is transmitted to the cognizant State Contract Manager, AOA. This includes all workplans, review checklists, Deliverables Observation Review (DOR) reports, and draft and final Semi-Annual Review (SAR) reports. Final documents will likewise be delivered to the Federal OCSE by the IV&V Service Provider at the same time that they are submitted to the DCSE and AOA. Beyond submission of contract deliverables, the IV&V Service Provider is otherwise prohibited from debriefing, consulting, providing presentations, delivering training sessions, or other information dissemination activities related to any IV&V Review Report finding or recommendation without the explicit approval of the AOA and the Federal OCSE. C. Standards to be Followed and Definition of Terms To meet the requirements of this RFP, the services shall be performed in accordance with the Institute of Electrical and Electronics Engineers (IEEE) Standard 1012-2004; other applicable, lifecycle-appropriate IEEE Standards (e.g., 12207 Software Life Cycle Process; 703 Software QA Plans; 1074 Developing Software Project Lifecycle Process; 828 Configuration Management Plans; and 830 Requirement Specifications, etc., to name a few) in assessing the State’s DACSES Replacement Project. Further, the IV&V Service Provider will employ the Capability Maturity Model Integrated (CMMI), and the Project Management Institute’s Project Management Body of Knowledge (PMBOK) Third Edition, and the PMBOK - Government Extension, as additional standards by which to assess the DACSES Replacement Project. All terms are as defined in the Federal Acquisition Regulation (FAR) 48 CFR Chapter 1 Part 2 - Definitions of Words and Terms, including amendments effective as of Federal Acquisition Circular FAC 97-25, May 02, 2001. D. Deliverables IV&V Service Provider shall provide the following Deliverables during this Contract: 1. An IV&V Workplan, in which IV&V Service Provider shall include Contract plans, schedules, and milestones for conducting all Contract tasks, assignment of appropriate staff resources, and deliverables. The IV&V Project schedule shall be created in a Microsoft ProjectTM -compatible format. 2. A complete set of Checklists to be used in the performance of reviewing key process areas within each project phase as well as across phases. The Checklists will be compiled based on the industry standards to be used on this project, including, but not limited to, the IEEE, CMMI, and PMBOK. 3. An in-depth Initial Review Report, in which IV&V Service Provider shall evaluate all areas of the DACSES Project described in this Solicitation’s Scope of Work. IV&V Service Provider shall provide Project context and quantitative data on each area analyzed, and include detailed recommendations on how DCSE and the DACSES Project can improve its processes, including which steps will be most beneficial to take next. 4. IV&V Service Provider shall include ACF Priorities to indicate the importance that the Federal OCSE considers the probability and severity of the subject finding’s impact on the DACSES Project. 5. Subsequent Semi-Annual Review Reports shall be in-depth and shall evaluate and quantify DCSE efforts to address the findings and recommendations in the previous Review Report(s). IV&V Service Provider shall include a tool to monitor DCSE’s progress toward resolution of Contractor findings and recommendations. The Contractor will also include Federal Priorities to indicate the importance that the federal offices consider the probability and severity of the subject finding’s impact on the DACSES Project. 6. Monthly Status Reports, generated during the performance of each Semi- Annual Review, shall describe the ongoing status and progress of each IV&V review, as well as contract financial information, including but not limited to: hours and associated costs expended in the reporting period and review-to- date, and cumulative contract-to-date. 7. Management Briefings, also known as a Debriefing, which shall be a formal presentation by the Contractor to the DCSE to discuss the findings and recommendations of the most recent IV&V Review Report, and shall occur no less than five business days after issuance of any respective Report in final. TABLE 13 - DELIVERABLES TASK DELIVERABLE TIME PERIOD Develop IV&V WorkPlan IV&V WorkPlan IV&V Service Provider shall complete the IV&V WorkPlan within 30 calendar days after the date Contractor is notified of Award of Contract. Develop IV&V Checklists IV&V Checklists Contractor will compile a comprehensive set of software development life cycle phase-specific checklists to be used in the performance of each IV&V Review. Each subset of these checklists, with their incorporated findings and artifacts, used in performing each Semi-Annual Review, will be included in the respective Review Report produced. Conduct Initial Review Initial Review Report Contractor's Initial Review shall commence within 10 calendar days after the date Contractor is notified of State acceptance of the IV&V WorkPlan. Contractor's Initial draft Review Report shall be delivered 90 calendar days after the review commences, but no more than 120 calendar days after the date Contractor is notified of Award of Contract. Contractor will incorporate State and OCSE comments, and deliver a final Initial Review Report no more than 30 calendar days after issuance of the Review Report in draft. Conduct Semi-annual Reviews Semi-annual Review Report Contractor's Semi-Annual Reviews shall be conducted continuous during the life of the Contract. Contractor's Semi-Annual Review Reports shall be delivered 90 calendar days after the respective review commences. Contractor’s Semi-Annual Reports will be prepared and presented approximately every six months. Contractor will confer with the Federal office prior to delivering all Review Reports to the State, and update these Reports, as needed, based upon Federal input. Contractor will update Reports, as needed, to incorporate State responses. Reports do not require a State response. State responses, such as a disagreement with or argument against a particular finding, will be appended to the respective Report Edits to Report content are only allowed for mistakes of fact and typographical errors. Conduct Monthly Status Report Monthly Status Report IV&V Service Provider shall submit Monthly Status Reports within 5 business days following the end of each calendar month that the Contractor is performing a Semi-Annual IV&V Review. Conduct Management Briefing Management Briefing, formal presentation and discussion on the most recently issued IV&V Report IV&V Service Provider shall prepare and deliver the Management Briefing, also known as a Debriefing, no less than five business days after issuance of any respective Report in final. Final Report FR A final report of IV&V tasks performed throughout the term of the contract, including a narrative description of "Lessons Learned", "Best Practices", and an opinion of the challenges and opportunities faced by the State with maintaining and operating its new automated system. Archive Documents ARCH A complete CD archive of all IV&V Documents including draft and final reports, status briefings, exception reports, all versions of the IV&V WorkPlan, IV&V Checklists, Deliverable Observation Reports, and Final Report. To be submitted with the invoice for the final Semi-Annual Review Report. Contractor may submit an Escalation Report to AOA, for the following: 1. Findings and recommendations from previous Initial and Semi-Annual Review Reports that have not been addressed. 2. Serious disputes about the accuracy, relevance, or applicability of specific findings and recommendations that are not resolved before the next Semi- Annual Review. 3. New findings that are so critical in nature that waiting to address them in the next Semi-Annual Review Report could have a significant, negative impact upon the DACSES Project. The State reserves the right to request additional analyses, called a Deliverable Observation Report (DOR), as needed. These DOR’s are highly focused reviews of specific project deliverables, technical analyses, or performance within key process areas within the project, and when required, will require their own separate scope statements to define them. Likewise, the IV&V Service Provider may suggest development of additional deliverables in specific areas. The State must authorize the need for any additional deliverables prior to their development. Where applicable, the deliverable must be developed in accordance with CMMI, PMBOK, and IEEE (or substantially and acceptably similar) standards. When no applicable standard exists, the methodology and processes used in the analysis and creation of the deliverable must be delivered to the Federal OCSE and the AOA prior to its use, and described in the final deliverable. All deliverables, standards, processes, plans, and applicable reference materials will be made available upon request of the State. The AOA and the Federal OCSE will review and approve the Initial and Semi- Annual Review Report formats, as well as all IV&V checklists, prior to their use. Copies of all deliverables will be delivered simultaneously to the Federal OCSE and to the AOA. Frequencies of deliverables are provided in the Statement of Work in Appendix A. The Federal OCSE and the AOA reserves the right to extend the due date, if appropriate, due to document size, schedule or changes in scope. The IV&V Service Provider must notify the Federal OCSE and the AOA of an anticipated delay of a deliverable, as far in advance of the due date as possible. All deliverables shall be approved by the Federal OCSE and the AOA in order for the task which produced them to be considered complete. In all cases, payments to the IV&V provider shall be contingent upon the Federal OCSE and the AOA approval of deliverables. No review will be considered complete until the approved documentation is delivered to and reviewed by the cognizant Federal OCSE and the AOA. For instances wherein the IV&V Service Provider delivers a one-time, focused, whether solicited by the Federal OCSE and the AOA or unsolicited, the proposal for review of some project artifact, process, or deliverable, called a Deliverable Observation Review (DOR), must include descriptions of the actions that shall be taken to produce the DOR Report, a proposed format and content outline for each DOR deliverable, and obtain the Federal OCSE and the AOA approval prior to any commencement of work. The AOA must approve, in writing, changes to milestones, deliverables or other material changes to the contract prior to implementation of changes. The AOA may require concurrence of the Federal OCSE in any changes prior to their implementation. E. Special Considerations All deliverables, reports, analyses, etc., whether in draft or final, must be delivered by the IV&V Service Provider directly to the Federal OCSE at the same time they go to the AOA. In this regard, in addition to the Federal OCSE, the IV&V Service Provider should ensure delivery to the AOA who is the State entity responsible for IV&V deliverables dissemination to the State’s Project, Agency, Departmental, and Stakeholder personnel. The AOA may not modify, or reject any IV&V Review Report beyond recommendations to emend mistakes of fact. AOA comments to all IV&V Review Report findings will only be appended to the respective report. For each area evaluated, the report should contain the current status of the State's effort, including any pertinent historical background information. The report should also contain a detailed analysis of each area, which answers, at the least, the following general questions: * What is under review, and what industry standards are applicable? * What is the State's current process in this area? * What's good about the State's process? * What about the State's process needs improvement and does the technology need improvement? * Is the State making measurable progress in this area? * Is the effort within the triple constraints of budget, scope, and schedule? * What standards is the project following (State, industry [IEEE, SEI, ISO, etc.,]) internally? * Is the appropriate documentation and other project artifacts accurate and up-to-date? * Is there adequate Stakeholder involvement in the Project? Are best practices and metrics employed to identify issues, progress, performance, etc? What risks to the project exist now, or can be projected in the near term? In the long term? Responses should be quantified whenever possible. The report should also contain detailed recommendations in each area specifying what can be done immediately and in the long term to improve the State's operation. Any technologies, methodologies, or resources recommended should reflect industry standards and be appropriate for the unique circumstances and constraints of the DACSES Project. The recommendations should also specify a method of measuring the State's progress against the recommendations. Follow-up reports should have quantified information on the progress that the State has made against the recommendations from the previous review. The follow-up report should also contain any additional and/or modified recommendations at the same level of detail as the initial recommendations. All report findings and recommendations should be historically traceable (with a clear and consistent method of identification/numbering) from the time they are first reported by the IV&V Service Provider until closure. All initial and follow-up reports will also include a brief section entitled "ACF Priority" at the conclusion of each finding and recommendation. The ACF Priority section will present the opinion of the Federal OCSE as to the respective finding’s probability of impact, and severity of impact. A sample. The Contractor will initially prepare this section for each open finding, and the Federal OCSE will modify its content accordingly. Appendix C presents a prioritization matrix and an example of the format of an ACF Priority section. The deliverables for this contract shall be provided in hardcopy form and on electronic media, using the following software standards (or lower convertible versions): DOCUMENT TYPE FORMAT Word Processing Microsoft Word 2003, or newer Spreadsheets Microsoft Excel 2003, or newer Graphics Microsoft PowerPoint 2003, or newer Project Management Microsoft Project 2003, or newer F. Contract Documentation Retention and Access to Contract Documentation All contract documentation and reports must be retained, at the IV&V Service Provider's expense, for a minimum of three years, unless the firm is notified, in writing, by AOA of the need to extend the retention period. The IV&V Service Provider will be required to make contract documentation available, upon request, to the following parties or their designees: * Office of Auditor of Accounts * Department of Health and Social Services, Division of Child Support Enforcement * Federal Office of Child Support Enforcement, Division of State and Tribal Systems or its designee * Parties designated by the federal or state governments or by AOA as part of a contract quality review process III. DESCRIPTION OF THE GOVERNMENT A. Location of the Work The Contractor’s location of the work will be within a 10-mile radius of the DCSE office at 84A Christiana Road, New Castle, DE 19720. IV. TIME REQUIREMENTS A. Proposal Calendar The following is a list of key dates up to and including the date proposals are due to be submitted: * Request for proposal issued January 8, 2010 * Pre-proposal conference January 21, 2010 * Proposal Due February 4, 2010 B. Date Contract May Commence The work may begin after the contract is fully executed and a State of Delaware purchase order is established by the DCSE. C. Contract Schedule The IV&V Contract activities and deliverables are listed in Appendix A. D. Entrance and Exit Conferences At a minimum, the IV&V Service Provider will hold an entrance and exit conference with the State DCSE’s key personnel. The IV&V Service Provider will also provide a scheduled debrief to DCSE on each semi-annual report published to the State in final, allowing for clarification and discussion of the respective report’s findings and recommendations. There will also be semi-annual meetings held with the Federal OCSE to brief that agency on the basis and justification for each respective report’s findings and recommendations prior to their publication in final. V. ASSISTANCE TO BE PROVIDED TO THE CONTRACTOR AND REPORT PREPARATION A. Assistance and Information to be Provided From the State * Workspace for up to three contractor staff will be provided while on-site at the project for the duration of the contract. The workspace will include desk or tables, phone and access to the projects’ LAN. The contractor is expected to have regular office space separate from the DACSES Replacement Project site. * Access to DACSES Replacement Project information, including, but not limited to, technical documentation and DACSES Replacement Project status data. * Access to State and contractor project personnel for information related to the project. * The State is not responsible for providing clerical or administrative support to the IV&V Service Provider. VI. PROPOSAL REQUIREMENTS A. General Requirements 1. Mandatory Pre-proposal Conference Firms interested in submitting a proposal must attend a mandatory pre- proposal conference at 1:00 p.m. on January 21, 2010 at the Division of Child Support Enforcement, 84A Christiana Road, New Castle, DE 19720, in Room 116. Both verbal and written questions will be accepted during the conference. Representatives of the DCSE and Federal OCSE will be available at the conference to discuss the services needed for the DACSES project. 2. Inquiries Inquiries concerning the RFP and the subject of the RFP must be made to: * Robert Hicks, CPA, PMP, Deputy Auditor 302-577-5264 (Robert.Hicks@state.de.us) * Candace Casto, CGFM, Senior Manager, Contracts & Administration 302-857-3910 (Candace.Casto@state.de.us) Direct contact with anyone other than the above-referenced contacts regarding this RFP is expressly prohibited without prior consent. Firms contacting State of Delaware or AOA employees risk elimination of their proposal from further consideration. Exceptions exist only for organizations currently doing business in the State of Delaware who require contact in the normal course of business. 3. Submission of Proposals To be considered, all proposals must be submitted by the due date listed in Section IV, A. to AOA’s third floor office, Townsend Building, Suite 1, 401 Federal Street, Dover, DE 19901, in writing and respond to the items outlined in this RFP using the following required format. AOA reserves the right to reject any non-responsive or non- conforming proposals, as well as proposals received after the specified date and time. By submitting a proposal, and in consideration of AOA’s efforts and representations with respect to the review and evaluation of its proposal, the firm expressly agrees to be bound by the terms and conditions of this RFP in the event it is selected for the engagement. The following material is required to be received by the due dates listed for a firm to be considered. B. Technical Proposal The purpose of the Technical Proposal is to demonstrate the qualifications, competence, and capacity of the firms seeking to undertake an independent IV&V services for the DCSE’s replacement of the DACSES system in conformity with the requirements of this RFP. As such, the substance of proposals will carry more weight than their form or manner of presentation. The Technical Proposal should list the particular staff to be assigned to this engagement, and specify an approach that will meet the RFP requirements listed in Appendix A. THERE SHOULD BE NO DOLLAR UNITS OR TOTAL COSTS INCLUDED IN THE TECHNICAL PROPOSAL DOCUMENT. The Technical Proposal should address all the points outlined in the RFP (excluding any cost information which should only be included in the sealed dollar cost bid). The Proposal should be prepared simply and economically, providing a straightforward, concise description of the proposer's capabilities to satisfy the requirements of the RFP. While additional data may be presented, the following subjects, items Nos. 2 through 10, must be included in the required format. Those subjects represent the criteria against which the proposal will be evaluated. 1. Required Format – A master copy (so marked) of a sealed Technical Proposal and four copies (total of five copies) to include the following. The RFP reference number (RFP10-CPA-01) must be noted on the outside of the sealed envelope along with "Technical Proposal." a. Title Page Title page showing RFP subject; RFP reference number; the firm's name; the contact’s name, address, telephone number, and e-mail address; and the date of the proposal. The transmittal letter must also clearly state and justify any exceptions to the requirements of the RFP that the applicant may have taken in presenting the proposal. b. Table of Contents Table of contents should clearly identify each section by page number. c. Transmittal Letter A signed letter of transmittal briefly stating the proposer's understanding of the work to be done, the commitment to perform the work within the time period, a statement why the firm believes it is best qualified to perform the engagement, and a statement that the proposal is a firm and irrevocable offer for one month after the due date of proposal. d. Detailed Proposal The proposal must conform to the proposal requirements of the RFP. AOA specifically reserves the right to waive any informalities or irregularities in the proposal format. The detailed proposal should follow the order set forth in Sections VI B and C of this RFP. 2. Mandatory Criteria (all criteria must be met or the proposal cannot be considered) a. Independence The firm should provide an affirmative statement that it is independent of the DHSS, DCSE, and DACSES Project. Independence is defined as having no current contractual relationship with any of the aforementioned government entities, as having no past contractual relationship such that work products or activities were created in support of the DACSES Project, and that all individuals proposed to this engagement have no current or historical employment relationship to or on the DACSES Project. The firm should also list and describe the firm's (or proposed subcontractors') professional relationships involving the DHSS, DCSE, and DACSES Project for the past five years, together with a statement explaining why each such relationship does not constitute a conflict of interest relative to performing the proposed services. In addition, the firm shall give the DCSE, Federal OCSE, and AOA written notice of any professional relationships entered into during the period of this agreement. b. License to Practice in Delaware An affirmative statement should be included that the firm and the engagement partner and manager/supervisor in charge are currently licensed or in the process of obtaining a license to work in the State of Delaware. 3. Firm Qualifications, Experience, and Requirements The proposer should state the size of the firm, the size of the firm's staff, the location of the office from which the work on this engagement is to be performed, and the number and skill sets of the professional staff to be employed in this engagement on a full-time basis, and the number and skill sets of the staff to be employed on a part-time basis. If the proposer is a joint venture or consortium, the qualifications of each firm comprising the joint venture or consortium should be separately identified and the firm that is to serve as the principal contractor should be noted, if applicable. The firm is also required to identify and include proof of any applicable professional credentials and certifications held by each individual proposed by the Contractor to this contract. Applicable credentials and certifications include, but are not limited to: Project Management Professional (PMP), Program Management Professional (PgMP), Six Sigma, and ITIL Professional. The firm shall certify that it has not been suspended or debarred from performing government contracts or IV&V services. In addition, the firm shall certify that it has not been the subject of any disciplinary action or inquiry in any jurisdiction during the past three years. The firm must certify that it shall maintain such insurance as will protect against claims under Worker’s Compensation Act and from any other claims for damages for personal injury, including death, which may arise from operations under this Contract. The Contractor is an independent contractor and is not an employee of the State. The IV&V Service Provider shall, at its expense, carry insurance of minimum limits as follows. The IV&V Service Provider shall provide evidence of such insurance. a. Comprehensive General Liability……….…. $1 million per claim/ $3 million aggregate b. Professional Liability……………...…….…. $1 million per claim/ $3 million aggregate 4. Partner, Supervisory, and Staff Qualifications and Experience Identify the principal supervisory and management staff, including engagement partners, managers, other supervisors and specialists, who would be assigned to the engagement. Indicate whether each person is registered or licensed to practice as a certified public accountant or a project management professional in the State of Delaware. Provide information on the project management experience of each person, including information on relevant continuing professional education for the past three years and membership in professional organizations relevant to the performance of these IV&V services and whether or not that person, within the past three years, has been the subject of any disciplinary action or inquiry in any jurisdiction. Provide as much information as possible regarding the number, qualifications, experience and training, including relevant continuing professional education, of the specific staff to be assigned to this engagement. Indicate how the quality of staff over the term of the agreement will be assured. The proposer should identify the extent to which staff to be assigned to the contract reflect the State of Delaware's commitment to Affirmative Action. Engagement partners, managers, other supervisory staff and specialists may be changed if those personnel leave the firm, are promoted or are assigned to another office. These personnel may also be changed for other reasons with the express prior written permission of AOA. However, in either case, the DCSE, Federal OCSE, and AOA retains the right to approve or reject replacements and the replacements must have substantially the same or better qualifications or experience. Consultants and firm specialists mentioned in response to this RFP can only be changed with the express prior written permission of AOA, which retains the right to approve or reject replacements. Other staff personnel may be changed at the discretion of the proposer provided that replacements have substantially the same or better qualifications or experience. 5. Prior Engagements With AOA List separately all engagements within the last three years, ranked on the basis of total staff hours, for AOA by type of engagement (i.e., audit, management advisory services, other). Indicate the scope of work, date, engagement partners, total hours, the location of the firm's office from which the engagement was performed, and the name and telephone number of the principal client contact. 6. Similar Engagements With Other Government Entities For the firm's office that will be assigned responsibility for the contract, list the most significant engagements (maximum of 5) performed in the last three years that are similar to the engagement described in this RFP. These engagements should be ranked on the basis of total staff hours. Indicate the scope of work, date, engagement partners, total hours, and the name and telephone number of the principal client contact. 7. Specific Engagement Approach The proposal should set forth a work plan, including an explanation of the methodology to be followed, to perform the services required in Section II of this RFP. In developing the work plan, reference should be made to such sources of information as the project's budget and related materials, organizational charts, manuals and programs, and financial and other management information systems. Proposers will be required to provide the following information on their engagement approach in accordance with the Statement of Work in Appendix A. 8. Identification of Anticipated Potential Problems The proposal should identify and describe any anticipated potential problems, the firm's approach to resolving these problems, and any special assistance that will be requested from the DCSE. C. Cost Proposal Required Format – A master copy (so marked) of a sealed Cost Proposal and four copies (total of five copies) to include the following. The RFP reference number (RFP10-CPA-01) must be noted on the outside of the sealed envelope along with "Cost Proposal." The sealed envelope shall contain the name and address of the proposing firm and the sealed dollar bid shall include the following: 1. Total All-Inclusive Maximum Cost The sealed dollar cost bid should contain all cost information relative to performing the engagement as described in this RFP. The total all- inclusive maximum cost to be bid is to contain all direct and indirect costs including all out-of-pocket expenses for each year (see Appendix B). AOA will not be responsible for expenses incurred in preparing and submitting the sealed technical proposal or the sealed dollar cost bid. Such costs should not be included in the proposal. The first page of the sealed dollar cost bid should include the following information: a. Name of Firm. b. Certification that the person signing the proposal is entitled to represent the firm, empowered to submit the bid, and authorized to sign a contract with AOA. c. A Total All-Inclusive Maximum Cost for each year. 2. Rates by Partner, Specialist, Supervisory, and Staff Level Times Hours Anticipated for Each The second page of the sealed dollar cost bid must include a schedule of professional fees and expenses, presented in the format provided in the attachment (Appendix B), that supports the total all-inclusive maximum cost. The cost of Special Considerations described in Section II E of this RFP should be disclosed as separate components of the total all-inclusive maximum cost. 3. Out-of-Pocket Expenses Included in the Total All-Inclusive Maximum Cost and Reimbursement Rates Out-of-pocket expenses for firm personnel (e.g., travel, lodging, and subsistence) will be reimbursed at the prevailing rates used by the State of Delaware for its employees. All estimated out-of-pocket expenses to be reimbursed should be presented on the second page of the sealed dollar cost bid in the format provided in the attachment (Appendix B). All expense reimbursements will be charged against the total all-inclusive maximum cost submitted by the firm. In addition, a statement must be included in the sealed dollar cost bid stating the firm will accept reimbursement for travel, lodging, and subsistence at the prevailing rate used by the State of Delaware for its employees. 4. Rates for Additional Professional Services If it should become necessary for DCSE, Federal OCSE, and AOA to request the contractor to render any additional services to either supplement the services requested in this RFP or to perform additional work as a result of the specific recommendations included in any report issued on this engagement, then such additional work shall be performed only if set forth in an addendum to the Agreement between DCSE, Federal OCSE, and AOA and the firm. Any such additional work agreed to between DCSE, Federal OCSE, and AOA and the firm shall be performed at the same rates set forth in the schedule of fees and expenses included in the sealed dollar cost bid. 5. Manner of Payment Payments will be net 30 days from the date of invoice for services upon receipt and acceptance of deliverables in accordance with the firm's dollar cost bid proposal. Invoices shall cover a period of not less than a calendar month and must include the following: (1) name of engagement; (2) amount and if it’s a progress or final invoice; and (3) a breakdown of staff and hours for each. Ten percent of the contract amount will be retained until the Exit Conference has been held and the final deliverables have been reviewed and accepted by the Federal OCSE and AOA. CONTRACTOR agrees to bill AOA only for actual work performed and out-of-pocket expenses incurred during its performance of the engagement. CONTRACTOR must submit all invoices in writing or electronic format to AOA for approval. Invoices must include the name of the staff member who performed the work, the nature of work performed, the date(s) work performed, the number of hours worked and the corresponding hourly rate, in addition to the amounts, descriptions and dates of all out-of-pocket expenses for which reimbursement is requested. If approved, AOA will forward the invoice to DCSE for payment. D. Submission of Proposals All completed sealed proposals must be delivered in person, by mail, or by carrier service (e.g., FedEx, UPS, etc.) to the following AOA personnel at AOA’s third floor office, Townsend Building, Suite 1, 401 Federal Street, Dover, DE 19901. Please do not have the proposals delivered to the Division of Corporations in the basement of the Townsend Building. Technical Proposal Robert Hicks, CPA, PMP Deputy Auditor 302-577-5264 Robert.Hicks@state.de.us Cost Proposal Candace Casto, CGFM Senior Manager, Contracts & Administration 302-857-3910 Candace.Casto@state.de.us Any proposal submitted by mail shall be sent by either certified or registered mail to AOA’s third floor office, Townsend Building, Suite 1, 401 Federal Street, Dover, DE 19901. Any proposal received after the above date shall not be considered and shall be returned unopened. The proposing firm bears the risk of delays in delivery. E. Modifications to Proposals Any changes, amendments, or modifications to the proposal must be made in writing, submitted in the same manner as the original response and conspicuously labeled as a change, amendment, or modification to a previously submitted proposal. Changes, amendments, or modifications to proposals shall not be accepted or considered after the time and date specified as the deadline for submission of proposals. VII. EVALUATION PROCEDURES A. Evaluation Committee Proposals submitted will be evaluated by an Evaluation Committee selected by AOA, which may consist of AOA, Federal OCSE, and DCSE personnel. The Evaluation Committee may negotiate with one or more of the qualified firms during the same period and may, at its sole discretion, terminate negotiations with any and all firms at any time. B. Review of Proposals Proposals that do not meet or comply with the instructions of this RFP may be considered non-conforming and deemed non-responsive and subject to disqualification at the sole discretion of the Evaluation Committee. The Evaluation Committee will use a point formula during the review process to score proposals. All assignments of points shall be at the sole discretion of the Evaluation Committee. Each member of the Evaluation Committee will first score each technical proposal by each of the criteria described in Section VII C below. The full Evaluation Committee will then convene to review and discuss these evaluations and to combine the individual scores to arrive at an average technical score for each firm. At this point, firms with an unacceptably low technical score will be eliminated from further consideration. After the composite technical score for each firm has been established, the sealed dollar cost bid will be opened and additional points will be added to the technical score based on the cost bid. The maximum score for cost will be assigned to the firm offering the lowest total all-inclusive maximum cost. Appropriate fractional scores will be assigned to other proposers. AOA reserves the right to retain all proposals submitted and use any idea in a proposal regardless of whether that proposal is selected and regardless of any statement of contention by a proposer to the contrary. C. Evaluation Criteria Proposals will be evaluated using three sets of criteria. Only firms meeting the mandatory criteria will have their proposals evaluated and scored for both technical qualifications and cost. The following represent the principal selection criteria that will be considered during the evaluation process. 1. Mandatory Elements All firms are required to meet these elements. No points assigned. a. The Offeror is independent and licensed to do business in Delaware. Please provide a copy of your firm’s State of Delaware Occupational License (issued by the Division of Revenue); a copy of the firm’s State of Delaware Board of Accountancy permit, if applicable; copies of the State of Delaware Board of Accountancy Permit of the engagement partner and the manager/supervisor in charge, if applicable; and copies of any other professional licenses, credentials and certifications of individuals proposed. b. Written affirmation that the firm has no conflict of interest with regard to any other work performed by the firm for the State of Delaware. c. The firm adheres to the instructions in this RFP on preparing and submitting the proposal. d. The firm has the following minimum limits in insurance: (1) Comprehensive General Liability….. $1 million per claim/ $3 million aggregate (2) Professional Liability…………….… $1 million per claim/ $3 million aggregate e. The firm has not been suspended or debarred from performing government contracts or from other governmental activity. f. The firm has not been the subject of any disciplinary action or inquiry during the past three years. 2. Technical Qualifications: (Maximum Points - [90]) a. Expertise and Experience (1) The firm's past experience and performance on comparable engagements. (2) The quality of the firm's professional personnel to be assigned to the engagement and the quality of the firm's management support personnel to be available for technical consultation. b. Contract Approach (1) Adequacy of proposed staffing plan for various segments of the engagement (2) Compliance with standards listed in Section I., A. 3. Cost COST WILL NOT BE THE PRIMARY FACTOR IN THE SELECTION OF AN IV&V FIRM Cost scores will be based on the number of proposals accepted and reviewed. The maximum score for cost will be assigned to the firm offering the lowest total all-inclusive maximum cost. Appropriate scores will be assigned to other proposers (e.g., if there are 10 proposals accepted and reviewed, then the firm with the lowest all-inclusive cost will be assigned 10 points, and the next lowest would be assigned 9 points, etc.). D. Reservation of Rights The Evaluation Committee reserves the right to: 1. Select for contract or for negotiations a proposal other than that with the lowest costs. 2. Reject any and all proposals or portions of the proposals received in response to this RFP or to make no award or issue a new RFP. 3. Waive or modify any information, irregularity, or inconsistency in proposals received. 4. Request modification to proposal from any or all contractors during the review and negotiation. 5. Negotiate any aspect of the proposal with any firm and negotiate with more than one firm at the same time. The Evaluation Committee reserves the right to reject any proposal from a firm who: 1. Has been convicted for commission of a criminal offense as an incident to obtaining or attempting to obtain a public or private contract or subcontract, or in the performance of the contract or subcontract; 2. Has been convicted under State or Federal statutes of embezzlement, theft, forgery, bribery, falsification or destruction of records, receiving stolen property, or other offenses indicating lack of business integrity or business honesty that currently and seriously affects responsibility as a State contractor; 3. Has been convicted or has had a civil judgment entered for a violation under State or Federal antitrust statutes; 4. Has violated contract provisions such as: a. Knowing failure without good cause to perform in accordance with the specifications or within the time limit provided in the contract; or b. Failure to perform or unsatisfactory performance in accordance with terms of one or more contracts; 5. Has violated ethical standards set out in law or regulation; 6. Any other cause listed in regulations of the State of Delaware determined to be serious and compelling as to affect responsibility as a State contractor, including debarment by another governmental entity for cause listed in the regulations. E. Oral Presentations During the evaluation process, the Evaluation Committee may, at its discretion, request any one or all firms to make oral presentations. Such presentations will provide firms with an opportunity to answer any questions the Evaluation Committee may have on a firm's proposal. Not all firms may be asked to make such oral presentations. All costs associated with participation in oral presentations conducted for the State of Delaware are the firm’s responsibility. F. Confidentiality of Documents All documents submitted as part of the firm’s proposal will be deemed confidential during the evaluation process. Proposals will not be available for review by anyone other than DCSE, Federal OCSE, and AOA and the Evaluation Committee or its designated agents. There shall be no disclosure of any firm’s information to a competing firm prior to award of the contract. The State of Delaware is a public agency as defined by state law, and as such, it is subject to the Delaware Freedom of Information Act, 29 Del. C. Ch. 100. Under the law, all the State of Delaware’s records are public records (unless otherwise declared by law to be confidential) and are subject to inspection and copying by any person. Vendor(s) are advised that once a proposal is received by the State of Delaware and a decision on contract award is made, its contents will become public record and nothing contained in the proposal will be deemed to be confidential except proprietary information. Firms shall not include any information in their proposal that is proprietary in nature or that they would not want to be released to the public. Proposals must contain sufficient information to be evaluated and a contract written without reference to any proprietary information. If a firm feels that it cannot submit its proposal without including proprietary information, it must adhere to the following procedure or their proposal may be deemed unresponsive and will not be recommended for selection. Firm(s) must submit such information in a separate, sealed envelope labeled “Proprietary Information” with the RFP number. The envelope must contain a letter from the firm’s legal counsel describing the documents in the envelope, representing in good faith that the information in each document is not "public record" as defined by 29 Del. C. § 10002(d), and briefly stating the reasons that each document meets the said definitions. G. Final Selection The AOA, DCSE, and Federal OCSE will select a firm based upon the recommendation of the Evaluation Committee. Neither AOA nor the Evaluation Committee nor the State of Delaware has any obligation to provide any firm with information or documentation related to the selection process and its decision with respect to this RFP beyond that which is described and required in this RFP or required by applicable law. H. Right to Reject Proposals Submission of a proposal indicates acceptance by the firm of the conditions contained in this RFP unless clearly and specifically noted in the proposal submitted and confirmed in the contract between AOA and the firm selected. AOA reserves the right without prejudice to reject any or all proposals. VIII. CONTRACT CONDITIONS The firm awarded the contract will be required to enter into a written agreement with AOA. AOA reserves the right to incorporate standard State of Delaware contractual provisions into any contract negotiated as a result of a proposal submitted in response to an RFP. Any proposed modifications to the terms and conditions of the standard contract are subject to review and approval by AOA. The firm shall be referred to as "CONTRACTOR" in the agreement. The selected firm will be expected to enter negotiations with AOA, which will result in a formal agreement between the parties. Procurement will be in accordance with the subsequent contracted agreement. This RFP and the selected firm’s response to this RFP will be incorporated as part of any formal agreement. If the firm to whom the contract is awarded fails to enter in the agreement as herein provided, the award will be annulled and an award may be made to another firm. Such firm shall fulfill every stipulation embraced therein as if they were the party to whom the first award was made. The selected firm under this RFP shall be subject to contractual provisions, in a form substantially similar to the following, as well as those agreed to by the parties and not specifically described herein. A. Term The term of the contract between the successful firm and AOA shall be determined by AOA with consideration of the agency's request. The agreement may be terminated by AOA at any time by giving written notice to the CONTRACTOR of such termination. Upon such termination, the CONTRACTOR will be paid for the hours of work actually completed. In the event the successful firm materially breaches any obligation under this Agreement, the IV&V Service Provider shall not be relieved of any liability to the State of Delaware for damages suffered by it by virtue of any such breach. AOA may withhold any payments to CONTRACTOR for the purpose of set-off for such damages. B. Preclusion from Resulting Contracts Any Contractor that has provided any consulting services or technical assistance that resulted in any specifications or concepts in this Solicitation, either directly or indirectly, is precluded from the award of a Contract pursuant to this Solicitation and is precluded from utilizing a Subcontractor that has provided any such services or assistance. C. Compensation AOA agrees to pay CONTRACTOR compensation calculated solely on the amount of work performed by firm’s staff, based on actual hours billed and hourly rates as set forth in Appendix B herein, as well as approved out-of-pocket expenses. CONTRACTOR agrees to bill AOA only for actual work performed and out-of-pocket expenses incurred during its performance of the engagement. CONTRACTOR must submit all invoices in writing or electronic format to AOA for approval. Invoices must include the name of the contractor who performed the work, the nature of work performed, the date(s) work performed, the number of hours worked and the corresponding hourly rate, in addition to the amounts, descriptions and dates of all out-of- pocket expenses for which reimbursement is requested. If approved, AOA will forward the invoice to the DCSE for payment. D. Non-Appropriation In the event the General Assembly fails to appropriate the specific funds necessary to enter into or continue the contractual agreement, in whole or in part, the agreement shall be terminated, as to any obligation of the State of Delaware requiring the expenditure of money for which no specific appropriation is available at the end of the last fiscal year for which no appropriation is available or upon the exhaustion of funds. E. Notice Any notice to AOA required under this Agreement shall be in writing and sent by regular mail and registered mail to: Attention: R. Thomas Wagner, Jr. Auditor of Accounts Office of Auditor of Accounts Townsend Building, Suite 1 401 Federal Street Dover, DE 19901 F. Formal Contract and Purchase Order The successful firm shall promptly execute an agreement incorporating the terms of this RFP within twenty days after the award of the contract. No bidder is to begin any service prior to receipt of a State of Delaware purchase order signed by two authorized representatives of the agency requesting service, properly processed through the State of Delaware Accounting Office and the (Agency). The purchase order shall serve as the authorization to proceed in accordance with the bid specifications and the special instructions, once it is received by the successful firm. G. Indemnification By submitting a proposal, the proposing firm agrees that in the event it is awarded a contract, it will indemnify and otherwise hold harmless the State of Delaware, its agents and employees from any and all liability, suits, actions, or claims, together with all costs, expenses for attorney’s fees, arising out of the firm’s, its agents and employees’ performance of work or services in connection with the contract, regardless of whether such suits, actions, claims or liabilities are based upon acts or failures to act attributable, in whole or in part, to the State of Delaware, its employees or agents. H. Compliance In performance of the contract the firm is required to comply with all applicable federal, state and local laws, regulations, policies, guidelines and requirements of the jurisdiction in which the Agreement is performed, as well as all applicable professional conduct rules and guidelines. The cost of permits and other relevant costs required in the performance of the contract shall be borne by the successful firm. The firm shall be properly licensed and authorized to transact business in the State of Delaware as provided in Section 2502 of Title 30 of the Delaware Code and other applicable statutes and regulations. I. Insurance The firm recognizes that it is operating as an independent contractor (and not an employee of the State of Delaware) and that it is liable for any and all losses, penalties, damages, expenses, attorney’s fees, judgments, and/or settlements incurred by reason of injury to or death of any and all persons, or injury to any and all property, of any nature, arising out of the contractor’s negligent performance under this contract, and particularly without limiting the forgoing, caused by, resulting from, or arising out of any act of omission on the part of the contractor in their negligent performance under this contract. The firm must certify that it shall maintain such insurance as will protect against claims under Worker’s Compensation Act and from any other claims for damages for personal injury, including death, which may arise from operations under the Agreement. The firm is an independent contractor and is not an employee of the State of Delaware. The firm shall, at its expense, carry insurance of minimum limits as follows: Comprehensive General Liability……..…… $1 million per claim/ $3 million aggregate Professional Liability……………..…....…. $1 million per claim/ $3 million aggregate The firm shall provide a certificate of insurance as proof that the firm has the required insurance. J. Non-Discrimination In performing the services subject to this RFP, the firm agrees that it will not discriminate against any employee or applicant for employment because of race, creed, age, marital status, religion, color, sex, genetic information, national origin, or disability. The successful firm shall comply with all federal and state laws, regulations and policies pertaining to the prevention of discriminatory employment practice. Failure to perform under this provision constitutes a material breach of contract. K. Covenant Against Contingent Fees The successful firm warrants that no person or selling agency has been employed or retained to solicit or secure this contract upon an agreement of understanding for a commission or percentage, brokerage or contingent fee excepting bona-fide employees, bona-fide established commercial or selling agencies maintained by the bidder for the purpose of securing business. For breach or violation of this warranty, AOA shall have the right to annul the contract without liability or at its discretion to deduct from the contract cost or otherwise recover the full amount of such commission, percentage, brokerage, or contingent fee. L. Contract Documents The RFP, the Purchase Order and the executed Agreement between AOA and the successful firm shall constitute the Contract between AOA and the firm. In the event there is any discrepancy between any of these contract documents, the following order of documents governs so that the former prevails over the latter: Agreement, Purchase Order, and RFP. No other documents shall be considered. These documents contain the entire agreement between AOA and the firm. Firm agrees to be bound by the terms of this RFP pending final execution of the Agreement by the parties. M. Applicable Law The laws of the State of Delaware shall apply, except where Federal law has precedence. The successful firm consents to exclusive jurisdiction in the State of Delaware and agrees that any litigation relating to this Agreement shall be filed and litigated in a court in the State of Delaware. N. Conflict of Interest In accordance with 45 CFR 307.15(b)(10)(ii), the Contractor and Subcontractors for the DACSES Project are prohibited from soliciting, proposing, or being awarded any project management, quality assurance, software design, development, or other manner of planning, design, development, or implementation phase activity on the DACSES Project, including the Title IV D agency (DCSE) and the IV-D agency's umbrella agency or department (DHSS), for which these IV&V services are being procured. This exclusion extends to any other State of Delaware project that may interact with or otherwise provide services to the DACSES Project during the life of this Contract, to ensure that the Contractor and Subcontractor are independent from the State, unless the State receives an exception from OCSE. IV&V Service Provider shall disclose any contractual relationship or any other contact with any State employee, Contractor, or Subcontractor involved in the development of Contractor's Response to the Solicitation resulting in this Contract. Any conflict of interest shall, at the sole discretion of the State, be grounds for rejection of Contractor's Bid or termination of this Contract. In addition to any requirements of law or through a professional code of ethics or conduct, Contractor is required to disclose any outside activities or interests that conflict or may conflict with the best interests of the State. IV&V Service Provider shall not plan, prepare, or engage in any activity that conflicts or may conflict with the best interests of the State during the life of this Contract without prior written approval of the State. Prompt disclosure is required under this paragraph if the activity or interest is related, directly or indirectly, to any person or entity currently under Contract with or seeking to do business with the State. O. Audits of, Access to, and Retention of Records IV&V Service Provider shall maintain adequate and separate accounting and fiscal records and account for all funds provided by any source to pay the cost of the Project. IV&V Service Provider shall permit an audit and/or examination of all such records, procedures, and accounts at any reasonable time by authorized representatives of appropriate federal and State entities. Authorized representatives shall have the right of access to any books, records, documents, accounting procedures, practices, or any other items of the Contractor that are pertinent to the performance or payment of this Contract in order to audit, examine, and make excerpts of records. Contractors employing two or more persons to supply services through a Contract that expends in excess of $500,000 or more in federal funds in one year must have a certified independent audit conducted in accordance with Government Auditing Standards and federal Office of Management and Budget (OMB) Circular A-133. Contractors receiving in excess of $50,000 per year in state or federal funds must have a certified independent audit of its entire operations, conducted in accordance with Generally Accepted Government Auditing Standards. The certified independent audit must cover the period for which the Contract was in effect. The financial statements must be prepared in accordance with Generally Accepted Accounting Principles and the report must include a Supplementary Schedule of State Awards listing all State revenues and expenditures by Contractor and a Supplementary Schedule of Revenue and Expenditures by function and funding source. A Certified Public Accountant or public accountant who has a valid and current permit to practice accountancy must perform the audit. The State retains the right to approve the selection of and examine the work papers of said auditor. No approval will be withheld unreasonably. At the request of the State, IV&V Service Provider shall provide for the examination of the books, records, documents, accounting procedures, practices, and any other items relevant to the Contract. If an audit is started before the end of the three-year records retention period, IV&V Service Provider shall maintain the records for three years from the date that all issues arising out of the audit are resolved or until the end of the three-year retention period, whichever is later. P. Scope of Agreement If the scope of any provision of this Agreement is too broad in any respect whatsoever to permit enforcement to its full extent, then such provision shall be enforced to the maximum extent permitted by law, and the parties hereto consent and agree that such scope may be judicially modified accordingly and that the whole of such provisions of the Agreement shall not thereby fail, but the scope of such provisions shall be curtailed only to the extent necessary to conform to the law. APPENDIX A PROCESS AREAS WITHIN THE SOFTWARE DEVELOPMENT LIFE CYCLE FOR INDEPENDENT VERIFICATION AND VALIDATION OF THE DACSES PROJECT A.1. Planning Oversight TABLE 2 - PLANNING OVERSIGHT TASK ITEM TASK # TASK DESCRIPTION Contract Verification PO-1 IV&V Service Provider shall evaluate and verify that the obligations of Contractor, Subcontractor, and external staff, including terms, conditions, statement of work, requirements, technical standards, performance standards, development milestones, acceptance criteria, and delivery dates, are clearly defined. IV&V Service Provider shall verify that performance metrics are included that allow tracking of Project performance and progress against the criteria set by the State. PO-2 IV&V Service Provider shall verify that the final Contract for Contractor team states that “IV&V Service Provider shall participate in the IV&V process,” for the purpose of being cooperative for coordination and communication of information. Feasibility Study PO-3 IV&V Service Provider shall perform ongoing evaluations and reviews, as needed, of DCSE methodologies used for the Feasibility Study, verifying the Feasibility Study is objective, reasonable, measurable, repeatable, consistent, accurate, and verifiable. PO-4 IV&V Service Provider shall review and evaluate PAPD(U)/IAPD(U) documents. PO-5 IV&V Service Provider shall review and evaluate the Cost Benefit Analysis. A.2. Project Management TABLE 3 - PROJECT MANAGEMENT TASK ITEM TASK # TASK DESCRIPTION Project Sponsorship PM-1 IV&V Service Provider shall evaluate and recommend improvements, as needed, to verify continuous executive agreement, participation, support, and commitment, and verify that open pathways of communication exist among all stakeholders. PM-2 IV&V Service Provider shall verify that DCSE management has signed off on all changes that impact Project objectives, cost, or schedule. Management Assessment PM-3 IV&V Service Provider shall evaluate and verify Project management and organization, to verify that lines of reporting and responsibility provide adequate technical and managerial oversight of Project. PM-4 IV&V Service Provider shall evaluate and report findings on Project progress, resources, budget, schedules, workflow, and reporting. PM-5 IV&V Service Provider shall evaluate and review coordination, communication, and management to verify stakeholders are working collaboratively and are following the Project Communication Plan. Project Management PM-6 IV&V Service Provider shall evaluate Project Management Plans and procedures to verify they are developed, communicated, implemented, monitored, and complete. PM-7 IV&V Service Provider shall evaluate Project Management Plans and Project Reports to verify Project status is accurately tracked using defined Project metrics. PM-8 IV&V Service Provider shall evaluate and verify that milestones and completion dates determined by DCSE are planned, monitored, and met. PM-9 IV&V Service Provider shall evaluate and verify that the tracking tool for Project issues and problems documents issues and problems as they arise; enables communication of issues and problems to appropriate stakeholders; documents a mitigation strategy as appropriate; and tracks issues and problems to resolution. Tracking issues and problems shall include, but is not limited to, technical and development efforts. PM-10 IV&V Service Provider shall evaluate the lifecycle development methodology(s), such as waterfall, evolutionary spiral, rapid prototyping, and incremental, to verify the methodology(s) is appropriate for the system being developed. Business Process Reengineering PM-11 IV&V Service Provider shall evaluate and verify Project capabilities and plans to redesign business systems to achieve improvements in critical measures of performance, such as cost, quality, service, and speed. PM-12 IV&V Service Provider shall evaluate and verify the DACSES Architecture Methodology has the strategy, management backing, resources, skills, and incentives required for effective change. Risk Management PM-13 IV&V Service Provider shall evaluate and verify that the Project Risk Management Plan is created and followed. IV&V Service Provider shall evaluate the Risk Management Plan and procedures to verify that risks are identified and quantified and that Mitigation Plans are developed, communicated, implemented, monitored, and complete. Change Management PM-14 IV&V Service Provider shall verify that the Project Change Management Plan is created and followed. IV&V Service Provider shall evaluate the Change Management Plan and procedures to verify they are developed, communicated, implemented, monitored, and complete; and that resistance to change is anticipated and prepared for. Communication Management PM-15 IV&V Service Provider shall verify that the Project Communication Plan is created and followed. IV&V Service Provider shall evaluate the Communication Plan and strategies to verify they support communications and work product sharing between all Project stakeholders, and verify Communication Plan and strategies are effective, implemented, monitored, and complete. Configuration Management PM-16 IV&V Service Provider shall evaluate and verify the Configuration Management Plan and procedures associated with the development process. PM-17 IV&V Service Provider shall evaluate and verify that all critical development documents, including, but not limited to, requirements, design, data scheme and code, are maintained under the required level of control. PM-18 IV&V Service Provider shall evaluate and verify that processes and tools are in place to identify code versions and rebuild system configurations from source code. PM-19 IV&V Service Provider shall evaluate and verify that all source and object libraries are maintained for training, testing, and production, and that formal sign-off procedures are in place for approving Deliverables. PM-20 IV&V Service Provider shall evaluate and verify that processes and tools are in place to manage system changes, including formal logging of change requests and review, prioritization, and timely scheduling of maintenance actions. PM-21 IV&V Service Provider shall evaluate and verify that mechanisms are in place to prevent unauthorized changes being made to the system and to prevent authorized changes from being made to the wrong version. Project Estimating and Scheduling PM-22 IV&V Service Provider shall evaluate and make recommendations on the estimating and scheduling process of the Project to ensure that Project budget and resources are adequate for the Work Breakdown Structure and schedule. PM-23 IV&V Service Provider shall review and evaluate schedules to verify that adequate time and resources are assigned for planning, development, review, testing, and rework. Project Staff PM-24 IV&V Service Provider shall evaluate the job assignments, skills, training, and experience of the staff involved in program development to verify that they are adequate for the development task. Project Organization PM-25 IV&V Service Provider shall verify that lines of reporting and responsibility provide adequate technical and managerial oversight of the Project. PM-26 IV&V Service Provider shall verify that the Project's organizational structure supports training, process definition, independent QA, Configuration Management, product evaluation, and any other functions critical to the Project's success. Subcontractors and External Staff, if any PM-27 IV&V Service Provider shall evaluate the use, in Project development, of Subcontractors or other external sources of Project staff, such as IT staff from another State of Oklahoma organization. PM-28 IV&V Service Provider shall evaluate and verify that the obligations of Subcontractors and external staff, including terms, conditions, statements of work, requirements, standards, development milestones, acceptance criteria, and delivery dates, are clearly defined. PM-29 IV&V Service Provider shall evaluate and verify that Subcontractor's software development methodology and product standards are compatible with DCSE standards and environment. PM-30 IV&V Service Provider shall verify that Subcontractors have and maintain the required skills, staff, plans, resources, procedures, and standards to meet Subcontractors’ commitment. IV&V Service Provider shall evaluate and verify the feasibility of any off-site support of the Project. PM-31 IV&V Service Provider shall evaluate and verify that any proprietary tools used by Subcontractors do not restrict the future maintainability, portability, and reusability of the DACSES System. DCSE Oversight PM-32 IV&V Service Provider shall verify that DCSE oversight is provided in the form of periodic status reviews and technical interchanges. PM-33 IV&V Service Provider shall verify that DCSE has defined the technical and managerial inputs the Subcontractor requires, including reviews, approvals, requirements, and interface clarifications, and has the resources to supply them on schedule. PM-34 IV&V Service Provider shall evaluate Project oversight to verify that DCSE, not Contractor, exercises ultimate responsibility for monitoring Project cost and schedule. A.3. Quality Management TABLE 4 – QUALITY MANAGEMENT TASK ITEM TASK # TASK DESCRIPTION Quality Assurance QA-1 IV&V Service Provider shall evaluate and make recommendations about the Project Quality Assurance Plan, procedures, and organization. QA-2 IV&V Service Provider shall verify that the QA organization monitors the fidelity of all defined processes in all phases of the Project. QA-3 IV&V Service Provider shall verify that the quality of all products produced by the Project is monitored by formal reviews and sign-offs. QA-4 IV&V Service Provider shall verify that Project self-evaluations are performed and that measures are continually taken to improve the process. QA-5 IV&V Service Provider shall monitor the performance of QA processes and/or designated staff by reviewing QA processes and reports, and by performing spot checks of system documentation. IV&V Service Provider shall evaluate findings and performance of the processes and reports. QA-6 IV&V Service Provider shall evaluate and verify that QA has an appropriate level of independence from Project management. Process Definition and Product Standards QA-7 IV&V Service Provider shall evaluate and make recommendations about all defined processes and product standards associated with system development. QA-8 IV&V Service Provider shall evaluate and verify that all major development processes are defined and that the defined and approved processes and standards are followed in development. QA-9 IV&V Service Provider shall evaluate and verify that processes and standards are compatible with each other and with system development methodology. QA-10 IV&V Service Provider shall evaluate and verify that all process definitions and standards are complete, clear, up-to-date, consistent in format, and easily available to Project staff. A.4. Training TABLE 5 – TRAINING TASK ITEM TASK # TASK DESCRIPTION User Training and Documentation TR-1 IV&V Service Provider shall evaluate and make recommendations about the training provided to system users. TR-2 IV&V Service Provider shall verify that user-friendly training materials are easily available to all users. TR-3 IV&V Service Provider shall verify that user-friendly Help Desk services are easily available to all users. TR-4 IV&V Service Provider shall verify that all required policy and processes, and related documentation, are easily available to users. TR-5 IV&V Service Provider shall verify that all training is provided on time, and is evaluated and monitored for effectiveness, with additional training provided as needed. Developer Training and Documentation TR-6 IV&V Service Provider shall evaluate and make recommendations about the training provided to system developers. TR-7 IV&V Service Provider shall evaluate and verify that developer training is technically adequate, appropriate for the development phase, and available at appropriate times. TR-8 IV&V Service Provider shall evaluate and verify that all required policy, process, and standards documentation is easily available to developers. TR-9 IV&V Service Provider shall evaluate and verify the required knowledge transfer occurs for maintenance and operation of the new system. TR-10 IV&V Service Provider shall evaluate and verify that all training is provided on time and is evaluated and monitored for effectiveness, with additional training provided as needed. A.5. Requirements Management TABLE 6 – REQUIREMENTS MANAGEMENT TASK ITEM TASK # TASK DESCRIPTION Requirements Management RM-1 IV&V Service Provider shall evaluate and make recommendations about the Requirements Management Plan processes and procedures for managing system requirements. RM-2 IV&V Service Provider shall evaluate and verify that system requirements are well-defined, understood, and documented. RM-3 IV&V Service Provider shall evaluate and verify the allocation of system requirements to hardware and software resources. RM-4 IV&V Service Provider shall evaluate and verify that software requirements can be traced through design, code, and test phases to verify that the DACSES System performs as intended and contains no unnecessary software elements. RM-5 IV&V Service Provider shall evaluate and verify that requirements are under formal configuration control. RM-6 IV&V Service Provider shall evaluate and verify Project's security for managing requirements and performing the risk analysis on each requirement. Security Requirements RM-7 IV&V Service Provider shall evaluate and verify that processes and equipment are in place to back up Project information and files and archive them safely at appropriate intervals. RM-8 IV&V Service Provider shall evaluate and make recommendations on Project policies and procedures for ensuring the system is secure and the privacy of client information is protected. RM-9 IV&V Service Provider shall evaluate and verify Project restrictions to ensure system and information access control. Requirements Analysis RM-10 IV&V Service Provider shall verify that the Requirements Analysis of DCSE and federal requirements and objectives has been performed to verify that DACSES System requirements are well understood, well- defined, and meet federal and state regulations. RM-11 IV&V Service Provider shall verify that major stakeholders have been consulted about the desired functionality of the DACSES System, and users have been involved in prototyping the user interface. RM-12 IV&V Service Provider shall verify that performance requirements, such as timing, response time, and throughput, meet user requirements. RM-13 IV&V Service Provider shall evaluate and verify that user's maintenance requirements for the DACSES System are completely specified. Interface Requirements RM-14 IV&V Service Provider shall evaluate and verify that all system interfaces are exactly described, by medium and by function, including input/output control codes, data format, polarity, range, units, and frequency. RM-15 IV&V Service Provider shall evaluate and verify approved interface documents are available and that appropriate relationships, such as interface working groups, are in place with all agencies and organizations supporting the interfaces. RM-16 IV&V Service Provider shall evaluate and verify that requirements specifications have been developed for all hardware and software subsystems in a level of detail to ensure successful implementation. System Modeling RM-17 IV&V Service Provider shall evaluate and verify that a well-defined Transition Plan and process for reengineering the system is in place and is followed, if a legacy system or a transfer system is or will be used in development. The process, depending on the goals of the reuse/transfer, may include reverse engineering, code translation, re-documentation, restructuring, normalization, and re-targeting. A.6. Operating Environment TABLE 7 – OPERATING ENVIRONMENT TASK ITEM TASK # TASK DESCRIPTION System Hardware OE-1 IV&V Service Provider shall evaluate current and projected system hardware configurations to verify that performance meets system requirements. OE-2 IV&V Service Provider shall evaluate and verify that hardware is compatible with existing DCSE processing environment, maintainable, and easily upgradeable. Evaluation and verification shall include, but is not limited to, CPUs and other processors, memory, network connections, bandwidth, communication controllers, terminals, printers, telecommunications systems (LAN/WAN), and storage devices. OE-3 IV&V Service Provider shall evaluate and verify current and projected outside vendor support of the hardware and DCSE hardware Configuration Management Plan and procedures. System Software OE-4 IV&V Service Provider shall evaluate current and projected system software to verify that capabilities meet system requirements. OE-5 IV&V Service Provider shall evaluate and verify that software is compatible with existing DCSE hardware and software environment, maintainable, and easily upgradeable. Evaluation and verification shall include, but is not limited to, operating systems, middleware, and network software, including communications and file-sharing protocols. Database Software OE-6 IV&V Service Provider shall evaluate current and projected database products to verify that capabilities meet system requirements. OE-7 IV&V Service Provider shall evaluate and verify the data format of the database is easily convertible to other formats, supports the addition of new data items, is scaleable, easily refreshable, and compatible with existing DCSE hardware and software, including any online transaction processing (OLTP) environment. OE-8 IV&V Service Provider shall review and evaluate the Project's process for administering the database, including backup, recovery, performance analysis, and control of data item creation. System Capacity OE-9 IV&V Service Provider shall evaluate the existing processing capacity of the system and verify that it is adequate for current statewide requirements for both batch and online processing. OE-10 IV&V Service Provider shall evaluate the historic availability and reliability of the system, including the frequency and criticality of system failure. OE-11 IV&V Service Provider shall evaluate the results of any volume testing or stress testing. OE-12 IV&V Service Provider shall evaluate any existing measurement and capacity planning program and shall evaluate the system's capacity to support future growth. OE-13 IV&V Service Provider shall make recommendations about changes in processing hardware, storage, network systems, operating systems, DACSES System software, and software design to meet future growth and improve system performance. A.7. Development Environment TABLE 8 - DEVELOPMENT ENVIRONMENT TASK ITEM TASK # TASK DESCRIPTION Development Hardware DE-1 IV&V Service Provider shall evaluate current and projected development hardware configurations to verify performance meets the requirements of system development. DE-2 IV&V Service Provider shall evaluate and verify that hardware is compatible with existing DCSE development and processing environment, maintainable, and easily upgradeable. Evaluation and verification shall include, but is not limited to, CPUs and other processors, memory, network connections, bandwidth, communication controllers, terminals, printers, telecommunications systems (LAN/WAN), and storage devices. DE-3 IV&V Service Provider shall evaluate and verify current and projected outside vendor support of the hardware. Development Software DE-4 IV&V Service Provider shall evaluate current and projected development software to verify capabilities meet system development requirements. DE-5 IV&V Service Provider shall evaluate and verify that software is compatible with the existing DCSE hardware and software environment, maintainable, and easily upgradeable. DE-6 IV&V Service Provider shall evaluate the environment as a whole to verify it shows integration. Evaluation shall include, but is not limited to, operating systems, network software, Computer-Aided Software Engineering (CASE) tools, project management software, configuration management software, compilers, cross-compilers, linkers, loaders, debuggers, editors, and reporting software. DE-7 IV&V Service Provider shall evaluate and verify current and projected outside vendor support of the software. A.8. Software Development TABLE 9 – SOFTWARE DEVELOPMENT TASK ITEM TASK # TASK DESCRIPTION High-Level Design SD-1 IV&V Service Provider shall evaluate and make recommendations on existing high-level design products to verify that the design is workable, efficient, and meets all system and system interface requirements. SD-2 IV&V Service Provider shall evaluate and verify that high-level design products conform to design methodology and standards. SD-3 IV&V Service Provider shall evaluate and make recommendations on high-level process, standards, methodologies, and CASE tools used. SD-4 IV&V Service Provider shall evaluate and verify that design requirements can be traced back to system requirements. SD-5 IV&V Service Provider shall evaluate and verify that all design products are under configuration control and formally approved before detailed design begins. Detailed Design SD-6 IV&V Service Provider shall evaluate and make recommendations on existing detailed design products to verify that the design is workable, efficient, and meets all high-level design requirements. SD-7 IV&V Service Provider shall evaluate and verify that detailed design products conform to design methodology and standards. SD-8 IV&V Service Provider shall evaluate and make recommendations on the detailed design and analysis process, standards, methodologies, and CASE tools used. SD-9 IV&V Service Provider shall evaluate and verify that design requirements can be traced back to system requirements and high-level design. SD-10 IV&V Service Provider shall evaluate and verify that all design products are under configuration control and formally approved before coding begins. Job Control SD-11 IV&V Service Provider shall evaluate and make recommendations on the existing job control and the process for designing job control. SD-12 IV&V Service Provider shall evaluate the system's division between batch and online processing to verify system performance and data integrity. SD-13 IV&V Service Provider shall evaluate batch jobs to verify appropriate scheduling, timing, and internal and external dependencies. SD-14 IV&V Service Provider shall evaluate and verify that job control language scripts are under configuration control. Code SD-15 IV&V Service Provider shall evaluate and make recommendations on the standards and process for code development currently in place. SD-16 IV&V Service Provider shall evaluate the existing code base to verify portability and maintainability, taking into account software metrics, including, but not limited to, modularity, complexity, and source and object size. SD-17 IV&V Service Provider shall evaluate code documentation to verify quality, completeness – including maintenance history, and accessibility. SD-18 IV&V Service Provider shall evaluate the coding standards and guidelines to verify Project's compliance with these standards and guidelines. Evaluation shall include, but is not limited to, structure, documentation, modularity, naming conventions, and format. SD-19 IV&V Service Provider shall evaluate and verify that developed code is under configuration control and is easily accessible by developers. SD-20 IV&V Service Provider shall evaluate and verify Project's use of software metrics in management and QA. Unit Test SD-21 IV&V Service Provider shall evaluate and make recommendations on the plans, requirements, environment, tools, and procedures used for unit testing system modules. SD-22 IV&V Service Provider shall evaluate the level of test automation, interactive testing, and interactive debugging available in the test environment. SD-23 IV&V Service Provider shall evaluate and verify the test process, that test results are verified, that the correct code configuration has been tested, and that the test is appropriately documented. A.9. System and Acceptance Testing TABLE 10 – SYSTEM AND ACCEPTANCE TESTING TASK ITEM TASK # TASK DESCRIPTION System Integration Test ST-1 IV&V Service Provider shall evaluate the plans, requirements, environment, tools, and procedures used for integration testing of system modules. ST-2 IV&V Service Provider shall evaluate the level of automation and the availability of the system test environment. ST-3 IV&V Service Provider shall verify that an appropriate level of test coverage is achieved by the test process, test results are verified, the correct code configuration has been tested, and tests are appropriately documented, including formal logging of errors found in testing. ST-4 IV&V Service Provider shall verify that the test organization has a level of independence from the development organization. Pilot ST-5 IV&V Service Provider shall evaluate the plans, requirements, environment, tools, and procedures used for pilot testing the DACSES System. ST-6 IV&V Service Provider shall verify that test scenarios are used to ensure comprehensive but manageable testing, and that tests are run in a realistic, real-time environment. ST-7 IV&V Service Provider shall verify that test scripts are complete, with step-by-step procedures, required pre-existing events or triggers, and expected results. ST-8 IV&V Service Provider shall verify that test results are verified, that the correct code configuration has been used, and that the test runs are appropriately documented, including formal logging of errors found in testing. Interface Testing ST-9 IV&V Service Provider shall evaluate interface testing plans and procedures for compliance with industry standards. Acceptance and Turnover ST-10 IV&V Service Provider shall verify that acceptance procedures and acceptance criteria for each product are defined, reviewed, and approved by program stakeholders prior to test, and that results of the test are documented. IV&V Service Provider shall verify that acceptance procedures address the process by which any software product that does not pass acceptance testing will be corrected. ST-11 IV&V Service Provider shall verify that acceptance testing based on the defined acceptance criteria is performed satisfactorily before acceptance of products. ST-12 IV&V Service Provider shall verify that the acceptance test organization has a level of independence from the Subcontractor or Contractor with which DCSE contracted to develop the DACSES System. ST-13 IV&V Service Provider shall verify that training of DCSE staff in using vendor-supplied software shall be ongoing throughout the development process. ST-14 IV&V Service Provider shall review and evaluate the Implementation Plan. A.10. Data Management TABLE 11 – DATA MANAGEMENT TASK ITEM TASK # TASK DESCRIPTION Data Conversion DM-1 IV&V Service Provider shall review and evaluate existing and proposed plans, procedures, and software for data conversion. DM-2 IV&V Service Provider shall verify that procedures are in place and are being followed to review the completed data for completeness and accuracy and to perform data clean-up as required. DM-3 IV&V Service Provider shall evaluate conversion error rates to verify the error rates are manageable. DM-4 IV&V Service Provider shall make recommendations to make the conversion process more efficient and to maintain the integrity of data during the conversion. Database Design DM-5 IV&V Service Provider shall evaluate new and existing database designs to verify they meet existing and proposed system requirements. DM-6 IV&V Service Provider shall recommend improvements to existing database designs to improve data integrity and system performance. DM-7 IV&V Service Provider shall evaluate the database design for maintainability, scalability, concurrence, normalization – where appropriate, and any other factors affecting performance and data integrity. A.11. Operations Oversight TABLE 12 - OPERATIONS OVERSIGHT TASK ITEM TASK # TASK DESCRIPTION Operational Change Tracking OO-1 IV&V Service Provider shall review and evaluate the DCSE statewide systems change request and defect tracking processes. OO-2 IV&V Service Provider shall evaluate implementation of the Operational process activities and request data to verify processes are effective and are being followed. Customer/User Operational Satisfaction OO-3 IV&V Service Provider shall review and evaluate user satisfaction with system and make recommendations for improvement. Operational Goals OO-4 IV&V Service Provider shall evaluate impact of the system on program goals and performance standards. Operational Documentation OO-5 IV&V Service Provider shall evaluate operational plans and processes. Operational Processes and Activity OO-6 IV&V Service Provider shall evaluate implementation of operational processes and activities, including backup, disaster recovery, and day- to-day operations, to verify the processes are being followed. APPENDIX C ACF PRIORITIZATION, PROBABILITY AND SEVERITY MATRIX The following table provides the guidelines to be used by OCSE in assigning priorities to the Independent Verification and Validation (IV&V) findings. The Overall Priority of each finding will be determined by first analyzing and rating the degree of impact the finding has on the project ("Degree of Project Impact"), the likelihood that the finding will occur ("Probability of Project Impact"), and the timeframe at which the finding is likely to require action by the State ("Time Criticality"). For example, if the degree of impact is rated HIGH, the probability of impact is MEDIUM, and the time criticality is LONG TERM, the overall priority of the finding would be MEDIUM, as shown on the fifth non-shaded row of the table. On the other hand, as time passes and the time criticality of the same finding changes from LONG TERM to SHORT TERM, the overall priority would change to HIGH, as shown on the third non-shaded row. A finding that receives an overall priority of URGENT must have resolution or show significant progress within three months of issuance of the IV&V progress review report. The State must provide evidence of this resolution or progress to OCSE prior to the next scheduled progress review. A finding that receives an overall priority of HIGH must have resolution or show significant progress prior to the next IV&V progress review. The State must provide evidence of this resolution or progress to OCSE prior to the next scheduled progress review. A finding that receives an overall priority of MEDIUM must have resolution or show significant progress during the next IV&V progress review. The State must provide evidence of this resolution or progress to OCSE during the next scheduled progress review. A finding that receives an overall priority of LOW must have resolution or show significant progress at the State's discretion. Any progress or resolution must be reported and evidence provided during a regularly scheduled progress review. Definitions of the rating factors follow the table. It should be noted that, while the OCSE recommendation is to fix URGENT and HIGH priority items first, there will likely be MEDIUM or LOW priority items that the State determines are easy and beneficial to fix. The State is encouraged to proceed to resolve these findings, as long as the schedule for resolving URGENT and HIGH priority findings is not significantly impacted. PRIORITY TABLE FOR IV&V FINDINGS OVERALL PRIORITY DEGREE OF PROJECT IMPACT PROBABILITY OF PROJECT IMPACT TIME CRITICALITY URGENT HIGH HIGH IMMEDIATE or SHORT TERM HIGH HIGH HIGH LONG TERM HIGH MEDIUM IMMEDIATE or SHORT TERM MEDIUM HIGH IMMEDIATE or SHORT TERM MEDIUM HIGH MEDIUM LONG TERM HIGH LOW ALL MEDIUM HIGH LONG TERM MEDIUM MEDIUM ALL MEDIUM LOW IMMEDIATE LOW HIGH IMMEDIATE LOW MEDIUM LOW SHORT TERM or LONG TERM LOW HIGH SHORT TERM or LONG TERM LOW MEDIUM ALL LOW LOW ALL DEFINITIONS Degree of Project Impact: High - Significant negative impact to cost, schedule, product quality, stakeholder acceptance, and/or other factors in the project. Medium - Moderate negative impact to cost, schedule, product quality, stakeholder acceptance, and/or other factors in the project. Low - Minimal impact to cost, schedule, product quality, stakeholder acceptance, and/or other factors in the project. Probability of Project Impact: High - Highly confident the negative impact will occur (> 80% certainty). Medium - Somewhat confident the negative impact will occur (> 50% certainty). Low - Uncertain if the negative impact will occur (< 50% certainty). Time Criticality: Immediate - Finding impacts the project now, or will impact the project or require resources within the next two months. Short Term - Finding will impact the project within the next six months. Long Term - Finding will impact the project at a future date greater than six months. All - Immediate, Short Term, and Long Term all result in the same overall priority rating. Page 1 of 55 11