100 likes | 206 Vues
Develop a tool to extract raw grade data from TTUMIS, preprocess files into SQL Server 2005 RDBMS, and extend framework for additional data sources as needed. Utilize open source tools for data collection and modeling. Ensure security using Apache, SSL, and encrypted connections.
E N D
Computer Science Retention Study • Data Collection Utility • Eric L. Brown
Purpose • Develop a tool to get raw grade data from TTUMIS for local processing. • Preprocess the raw text files into a SQL Server 2005 RDBMS. • Develop a framework that can be easily extended to include other data sources as required by the study. • Use as many open source tools as possible.
Parts of the Puzzle (Part 1 - The Process) • Raw Data Collection • Establishing Performance Metrics • Model Data via Established Metrics • Dissemination • Maintenance
Parts of the Puzzle(Part 2 - the helpers) • pexpect-2.1beta (pexpect.sf.net) • freetds-0.63 (freetds.org) • pymssql-0.7.4 (pymssql.sf.net) • PyXML-0.8.4 (pyxml.sf.net) • Gnosis_Utils-1.2.1 (gnosis.cx) • FCKEditor-2.2 (fckeditor.net)
Parts of the Puzzle(Part 4 - the scripts) • createtables.py / parse.py / samplereport.py • index.py / form_action.py • mathdata.py / getdata.py • dataeditor.py / dataeditorpost.py
Security • Security before all else... • Apache 1.3.33 / SSL • ssh / sftp • will have encrypted connection to MS-SQL • not on a public system
Conclusions • I love Python for a scripting language, but be aware of which version you are running. • There is more security work to be done. • Only small requests can be run through the browser window due to timeouts. • Web will be mostly for reporting in future releases.
Thank You! • Questions • Questions • No Question