IMPLEMENTING AN ELECTRONIC RECORDS MANAGEMENT PROGRAM

1. IMPLEMENTING AN ELECTRONIC RECORDS MANAGEMENT PROGRAM • Philip C. Bantin • Indiana University Archivist • bantin@indiana.edu • IU Electronic Records Program Website: http://www.indiana.edu/~libarch/ER/NHPRC-2/index.html

2. OUTLINE OF PRESENTATION • 1) Requirements for an Electronic Records Management System (ERMS) • 2) Why On-Line Transaction Processing (OLTP) Systems, Data Warehouses, and Electronic Document Management Systems (EDMS) Do NOT Meet These Requirements

3. OUTLINE OF PRESENTATION • 3) Capturing Records – Strategies • ** Business Process Engine – Example: European Registry System • ** Workflow or Routing Engine – Example: IU Workflow Engine • ** Records Management Applications (RMA) – Example: TRIM • 4) Capturing Records: Metadata Specifications

4. Information Systems • Systems Development Lifecycle • System concept: purpose, goals, scope • Analysis: user/functional requirements • Design • data design: what information? • software design: processed how? • interface design: user interaction? • Coding and testing: execute & evaluate • Key issue: Systems do (only) what they’re designed to – purpose, goals, scope, requirements.

5. Data and Information Systems • Transaction Processing • Data-oriented • Automate basic business processes • Decision Support • MIS/EIS • Data warehouse • Electronic Document Management Systems (EDMS)

6. Transaction Processing Systems (TPS) Employing DBMS Software • The most basic business system and the heart of most organizations • TPS is a computerized system that performs and records the daily routine transactions necessary to the conduct of business • Primary goal is to automate computing intensive business transactions, such as those undertaken in the financial and human resource functional areas

7. TRANSACTION PROCESSING SYSTEMS • Inputs=transaction data like financial information, human resource data • Processing=sorting, listing, updating, merging • Outputs=bills, paychecks, orders, detailed lists and reports

8. Transaction Processing • Data Representation and Storage • Sequential files • ordered set of structurally similar records • Indexed files • 2 part structure: data + index • Databases • multiple entities (tables) linked by key-fields • minimal redundancy

9. Data Systems and Record Systems • How are they Similar? • They both… • capture and store data • organize digital data • enforce standards for representation of data • protect data from destruction (accidental or intentional). • make data available for people to use (search, browse, retrieve, etc.)

10. But are TPS Good Recordkeeping Systems? • Transaction Processing • Primary Goal: to automate some activity or business process. • Transaction Processing and Records • Recordkeeping may be the business process • …or records may be the byproduct of the business process; but recordkeeping is not the primary purpose of the system • …or records may be difficult or impossible to locate or retrieve …if they exist there at all.

11. Transaction Processing • Record-oriented business process • General Ledger • keep record of debits and credits • cumulative history of transactions: account, object, amount, date… • Academic Record (Transcript) • keep record of courses, grades, degrees • cumulative history of academic career: • course department & title, instructor, grade • semesters and dates of enrollment • degree, school, major, honors, date

12. Transaction Processing • Records as byproduct • Hotel Reservation • retrieve room vacancy information • collect guest information • assign guest to room • record guest information & reservation dates • Employee Payroll • combine pay rates and hours worked • produce paychecks • record annual totals of earnings, withholdings, benefits • But managing complete records of all business transactions over the life cycle is not the primary objective of the system

13. Transaction Processing Systems and Record Systems • Why aren't TPS reliable Recordkeeping Systems? • They capture incomplete information. Records are made up of content, context, and structure. Many Information Systems capture only content, and not always completely. • They keep only current information. Many Information Systems store only "current values" for key bits of information and they do a bad job of tracking the history of changes to these values.

14. Transaction Processing Systems and Record Systems • Why aren't TPS reliable Recordkeeping Systems? • They scatter information. To avoid redundancy, Information Systems store each bit of information just once, often in separate tables or databases, and they rely on computer software to re-connect the bits of information when needed. Related pieces of information become disconnected over time, or may be preserved or discarded according to different timetables.

15. Transaction Processing Systems and Record Systems • However…in many cases TPS ARE (de facto) Record Systems. • An institution or agency may be using a TPS to conduct its business, and relying on the databases of that system to keep a record of key activities. • It may not be a really good Record System, but that Data System IS the institution's (agency's) primary system for keeping records. • Key issue: Is it possible to make an adequate Record System from a Transaction Processing System?

16. DECISION SUPPORT SYSTEMS • A computer-based information system that provides a flexible tool for analysis and help managers in making strategic decisions • Uses data and information to produce flexible, on-demand reports or information, and assistance in decisions about unstructured problems • By means of models, DSS can provide valid representations of real world systems

17. DATA WAREHOUSE • The data warehouse concept is about unbundling the two environments. In one environment – TPS - the business automates its processes on many different on-line transaction systems in the most effective and expedient manner possible. The data from these many and varied systems is then used to populate a database comprising all the data necessary to support decision making in a separate data warehouse environment.

18. DATA WAREHOUSE • Unlike operational databases that are set up to handle transactions and that are kept current as of the last transaction or update, data warehouses are analytical, subject oriented and are structured to aggregate transactions as a snapshot in time.

19. DATA WAREHOUSES - PRIMARY CHARACTERISTICS • Separate from the Operational systems and populated by data from these systems • Available entirely for the task of making information available to users • Time-stamped and associated with defined periods of time • Subject Oriented as defined by the customer • Accessible to users who have limited knowledge of computer systems

20. Data Warehouses as Recordkeeping Systems • Data Warehouses are NOT ERMS • Do not meet many of the requirements • 1) Do not routinely capture records – Evidence of Business Transactions – More concerned about transforming data into information • 2) Do not manage information or records over the entire life cycle

21. EDMS as Recordkeeping Systems • Many EDMS functions overlap with ERMS • EDMS functionality typically includes: indexing of documents, storage management, version control, integration with desktop applications, and retrieval tools to access the documents • But they are not fully functioning ERMS

22. EDMS As A Recordkeeping System • EDMS: Primary purpose is to support day-to-day use of documents for ongoing business • ERMS: Primary purpose is to provide a secure repository for authentic and reliable business records

23. EDMS AS RECORDKEEPING SYSTEMS • WHAT IS A DOCUMENT? • “A grouping of formatted information objects regardless of medium or form that can be accessed and used by a person.” • “A document is a container which brings together information from a variety of sources, in a number of formats, around a specific topic, to meet the needs of a particular individual.” • A document is discrete and identifiable - Structural unit of text like a report or letter or minutes of meetings

24. EDMS AS RECORDKEEPING SYSTEM • WHAT IS A RECORD? • A specific type of information produced by a business event • Evidence of business transactions • Evidence is metadata documenting the context of creation, and the content and structure of the record • Forms the basis for Recordkeeping Systems

25. EDMS As A Recordkeeping System • EDMS allows documents to be modified and exist in several versions • ERMS prevents records from being modified

26. EDMS As A Recordkeeping System • EDMS may allow documents to be deleted by owner of data • ERMS prevents records from being deleted except in certain strictly controlled circumstances

27. EDMS As A Recordkeeping System • EDMS may include some retention controls • ERMS must include rigorous retention controls

28. EDMS As A Recordkeeping System • EDMS may include a document classification scheme • ERMS must include a robust record classification scheme

29. EDMS As A Recordkeeping System • Most EDMS do not meet many of the requirements of a EDMS • 1) Do not routinely capture records – Evidence of Business Transactions • 2) Do not manage information or records over the entire life cycle

30. Strategies for Capturing Record Content and Record Metadata

31. Overall Goals of Record and Metadata Capture • System officially “captures” records for the University • Capture involves process of: • 1) Registering a record • 2) Deciding which class it should be classified to • 3) Adding further metadata to it • 4) Storing it in the ERMS.

32. STRATEGIES FOR CAPTURING RECORDS • 1) Applications that include BUSINESS PROCESS ENGINES • 2) Applications that include WORKFLOW ENGINES • 3) RECORDS MANAGEMENT APPLICATIONS (RMA)

33. Business Process Models • Primary Advantage: • Models clearly identify all activities within a process and consequently identify all records and the relationships between records and processes • It is a complete representation of the business process and of the various inputs and outputs generated by the process

34. Business Process Models • Primary Disadvantage • These models are not often designed into the system - automated business process engines are not the norm in North America • Good Examples of automated business process engines are the European Registry Systems

35. Registry System in Finland • All recordkeeping functions in a Finnish government agency are supervised by a recordkeeping schedule • Based on a hierarchical classification of agency functions, activities and business processes which serves • the registry system • filing • appraisal

36. Registry System - Operational Procedures • Registration is based upon a business process • e.g, filling an archivist’s position in the university archives • This business process gets a registry number that is based on the functional classification schema • uniquely identifies the business process in question • links the business process to the function it serves, in this example personnel management

37. Registry System - Operational Procedures • Each business process is registered step by step regardless of the boundaries of the organizational units that participate in the process • Records created by each step are filed in a document management system and are linked to registry entries

38. Example of Recordkeeping in Registry System - Filling an archivist’s position in the University Archives Date Transaction Agent 2001-04-24 an announcement of the vacant position sent Office A R to a local newspaper 2001-05-10 an application sent by Ms. X Office A R 2001-05-10 an application sent by Ms. Y Office A R 2001-05-12 an application sent by Mr. W Office A R 2001-05-24 applications sent for review to the archives Office A 2001-06-24 statement of the applicants sent to Office B Archives R after interviews 2001-06-30 a request for additional information from Office B R the archives 2001-07-10 a revision to a previous statement to Office B Archives R 2001-07-25 a decision to appoint Ms. Y as an archivist Office B R 2001-07-26 a letter to Ms. Y informing her that she has Archives R been appointed

39. Advantages of the Registry System • For records and archives management the registry links records to • business processes and functions that create them • other records created by the same business process and function • Provide contextual information and ensure the integrity of electronic records

40. Conceptual Design – Workflow Workflow is"the automation of a business process, in whole or part, during which documents, information or tasks are passed from one participant [human or machine] to another for action, according to a set of procedural rules.” http://www.e-workflow.org/ “Starting from creation and ingestion, we should integrate the workflow process with the preservation process: appraisal, verification, maintenance and, eventually, retirement.” Su-Shing Chen “The Paradox of Digital Preservation” Computer (IEEE Computer Society), March 2001

41. WORKFLOW MODELS • Advantages: • 1) Commonly used • 2) Coming back into fashion with an emphasis on life cycle management • 3) Often Automated

42. WORKFLOW MODELS • Disadvantages • 1) Primarily a Routing mechanism and thus: • a) not all records that are created go through workflow process • b) routing process may not identify all activities, all inputs and outputs within a given process; consequently some records within the process may not be identified and captured

43. UIS EDEN Workflow Engine Overview of workflow engine for IU’s OneStart portal.

44. Other HRMS SIS FIS IUIE Users Security Workflow OneStart & EDEN Component-Based Development User Interface OneStart Customized Personalized Adaptable Desktop Application Delivered Channels Applications Other Content Services Infrastructure Record Keeping Application Services EDEN

45. Goals of EWE • Service to enterprise applications that routes electronic transactions to individuals or systems for work, approval or notification. • Audit trail of all routing and actions taken on electronic transactions. • Integration with IU’s portal.

46. Technical Architecture • Component Based – EJB’s • Application interface • Application post-processors • Route Modules • Documents processed in XML format • Minimal content is EWE route control data • EWE can be used as the storage for pending transactions. • EWE can enable versioning of information. • Web services – applications required to register a web service for each document type.

47. Document creation Application using EWE • Applications send XML version of electronic document to EWE. • (Document must be of a registered type.) Workflow engine Request route ID for doc

48. Route Document Application using EWE • EWE submits XML to route modules Workflow engine RouteDoc( XML contents of doc )

49. Discover Action Requests Routing Modules Workflow engine 1) XML for document 2) ActionRequests Route modules scan XML for specific route controls and match document information to action request rules. 3) Add action requests to document route Route modules return list of action requests

50. Activate Next Action Request • Document’s action requests are activated by: • Route type priority as determined by document type route template • Request priority as determined by the route rule • Order in which request was associated with the document. • The activated requests are put on the appropriate action lists of the persons or processes.