Student Records Data Warehouse Project Notes of Meeting 29 May 2003 Present: Ray Burnley, Raymond Thompson, Sue Milward, Tom Rosenbloom ---------------------------------------------------------------------- Four options were presented to enable Raymond and Tom to download data easily from the proposed data warehouse. 1. Dump of data into CSV files 2. Dump of data into Oracle database 3. Dump of data into XML files 4. Dump of data into MySQL database (the underlying SAM3 database) The preferred option for Tom was to access the data from a MySQL database as he has independently invested heavily in Dreamweaver and PHP development to produce SHiPSS web pages. Raymond was flexible about the data source so long as he could produce a large flat file of his data. Ray stated that most of the code for downloading the data from SITS into MySQL is already written and that minimum further development would be required. IT Services also want to cause minimal extra work to the 'end users'. This requirement would support the MySQL option. There is a further requirement from SML and SHiPSS for production of lecture and tutorial timetables. This functionality is currently provided by the SAM3 application. If the SAM3 application is due to be phased out in Sept 2003, then an alternative process will have to be found for these schools. Raymond will require access to the data warehouse by mid-July. If MySQL is the data format, then SHiPSS have no deadline date as their software will not require any updating. Ray has produced an initial mapping of SITS and SAM3 data structures. ACTIONS Confirm that Ray will be allocated time to maintain the MySQL database for the next year and therefore that the MySQL option is viable. [Sue] Confirm server location of the warehouse database [Sue] Investigate alternate functionality for the SML/SHiPSS timetable production, or gain approval for maintenance of part of the SAM3 application by Ray. [Sue] Agree content of data fields to be downloaded. [Ray/Raymond/Tom] ====================================================================== Reponse from: Tom Rosenbloom Dated: 02-Jun-2003 ---------------------------------------------------------------------- For the record, re the data warehouse meeting and Sue's notes. Mainly, I want to refute the statement that my reasons for backing mySQL is that I have "independantly invested heavily in Dreamweaver and PHP development to produce SHiPSS web pages". That is partly the reason, but please see the attached. regards, Tom. ...................................................................... Data Warehouse -- mySQL? The case for a mySQL data warehouse (now and in future) is not purely based on my personal investment of time in PHP/mySQL/DreamweaverMX. I am also thinking of a greater good, namely reversing the trend towards de-centralisation and independent development of web solutions within the University. Sue said in the meeting that it is only Raymond T. and myself who are prospective clients of the data warehouse. In fact this is not true. There are many schools and departments around the University that have developed web applications of one sort or another, but they have gone the independent route -- running their own servers and in some cases even outsourcing development work and hosting on commercial servers. They have done this because they don't believe they will get the service and support they need from IT Services. In my case I have been keen to develop SHiPSS web applications and use of dynamic web content using centralised resources provided by IT Services -- I think this is the correct way of doing it from my point of view, and also from the institutional point of view. However, it has been easier for me because (1) I have connections with IT Services (2) SHiPSS uses (used) SAM (3) the school has good connections with IT Services etc. mainly due to Keith's activities. Sue also said in the meeting that, in the long run, there will be no need for schools/departments to develop web applications or use dynamic content because e- vision will do that for them. That may be true for applications directly concerned with student records etc., but (1) there will always be other applications that schools want to develop (2) schools may simply wish to use a bit of dynamic content in their ordinary web pages. Even if schools' web applications do duplicate some elements of IT Services projects, better that they are 'semi-centralised' rather than completely de-centralised, i.e. that they are based on data in the warehouse and run on a central server, rather than local data on an independent server. So, IT Services should be providing an environment in which schools can develop their own applications and use of dynamic content, but in as centralised -- hence efficient -- a manner as possible. So far to this end, ITS have installed PHP and mySQL on the central web server. PHP has become very much the standard tool for developing web applications and using dynamic content. (Owing not only to it being open source, platform independent and Apache-friendly, but also because it is "HTML-centric", and has "the capability to enable people to get simple things done extremely quickly" -- quotes from Rasmus Lerdorf, originator of PHP). That's the scripting side. Most web applications are based around a database. PHP can connect to any number of different databases, true, but mySQL has become its natural partner (mainly because it is open source, I think) hence that is what has been installed on the University web server. So if people like me are developing web applications or wish to use dynamic content they will probably want to use PHP/mySQL (especially if they also want to use Dreamweaver MX). Many University web applications will want to refer to data in the warehouse, and much dynamic content should also come from there. Technically it wouldn't be an insurmountable problem to connect to an Oracle database for warehouse data and use mySQL for 'local' data, but I believe that this complication would deter new clients. And, as we agreed, it is an unnecessary complication when most of the mySQL work has already been done by Ray B. t.m.rosenbloom@ex.ac.uk 02.06.03