1 / 33

Population file updates

Population file updates. Contents of the population file. Student background variables will be used for reporting purposes and to verify the representativeness of the sample . Course-related variables are required to develop the sample frame and to pre-populate sections of the survey .

uta
Télécharger la présentation

Population file updates

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Population file updates

  2. Contents of the population file • Student background variables will be used for reporting purposes and to verify the representativeness of the sample. • Course-related variables are required to develop the sample frame and to pre-populate sections of the survey. • Student address details will only be used to send letters to students who do not respond the email invitation.

  3. Derived variables • Age • Concurrent /major course indicator • Cumulative EFTSL since commencement • Enrolment status • Exclusions • Sample frame (set to blank in population file) • Defining commencing and final year students • Strata • Subject area

  4. Sample frame exclusions • Enrolled in a postgraduate or non-award course • Offshore undergraduates • Onshore undergraduates in middle year of a course • Onshore undergraduates enrolled concurrently in another course • Onshore undergraduates in strata with six or fewer students • Onshore undergraduates randomly excluded from very large strata

  5. Defining final year students • The ratio of ‘EFTSL completed successfully’ (E355) and ‘currently in progress’ (E339) to the total EFTSL for the course (E350) represents a student’s progression to date. • The standard solution adjusts for attendance mode (E330) and course duration (E350) and requires a greater proportion of cumulative EFTSL for longer courses. • Final year enrolment estimates for 2014 accord reasonably well with course completions in 2013

  6. Sample strata • The sampling strata were built on the 45 Subject Areas reported on the MyUniversity website. • The code ‘2236_LY_29’ refers to Curtin University of Technology (2236) where final year students (LY) were enrolled in the Subject Area Business Management (29). • Students in combined/double degrees are allocated to the Subject Area with the fewest students.

  7. What we need you to do The information required to conduct the survey has already been compiled - well almost. We need you to do four things: • Inspect the data file for correctness, but not forensically. • Update students’ current enrolment status to make sure we don’t contact students who are not currently enrolled. • Provide email addresses for all students (and mobile phone numbers if you want us to SMS). • Provide term addresses for onshore international students.

  8. What we need you to tell us about Were there any significant changes in your student profile between 2013 and 2014 that might affect subject areas and stages of enrolment? • Have different fields of education been allocated to the same courses in 2013 and 2014? • Are course duration (E350 Course of Study Load) values correct? • Cumulative EFTSL…. • Is there anything else we need to know about your data?

  9. Sampling

  10. (not so) Random terms… • Confidence level – the percentage of all possible samples that can be expected to include the true population parameter • Confidence interval – a measure of the reliability of an estimate (margin of error) • Sample factor – the number of records required to achieve an completed survey

  11. Sampling & percentages • The accuracy of an estimate also depends on the percentage of a sample that selects a particular answer. • In general, it is easier to be sure about extreme answers so if 95% of respondents agree with a statement, the chances of making an error are small. • If 51% of people agree then the changes are greater that an error will be made because of the general ambivalence in relation to the statement. • Sampling theory suggests that percentages be treated conservatively and estimates be made from the worst case, middle of the road scenario of 50%

  12. Previous approach to sampling • Focus on obtaining a 35% response rate • Estimates of the sample factor based on 2012 UES • Difficult to determine accurately due to sample frame, response rate and mode issues • Sampling focused at an institutional level not a strata (subject area) level • Large strata capped at 1,333 records irrespective of size

  13. 2014 approach to sampling • Focus on a response rate that supports reporting at a 90% confidence level +/- 7.5% • Required response rates are different for different strata. The smaller the subject area, the better the response rate needs to be. • Sample factor estimates informed by the 2013 UES allowing for better estimation of differential response rates across strata • For large strata, the required sample records depends on the number of students and the 2013 response rate

  14. Complexity… • This approach means that the sample factor is calculated separately for each strata (ie institution by subject area) taking into account whether the student is commencing or in their final year • Our assumptions are conservative estimating that responses to key items are 50% rather than the 70% to 80% we actually obtain in response to key items • We are also sampling on the basis of a stretch goal which based on a reporting standard of 90% confidence level +/- 5% (rather than +/-7.5%)

  15. What does this mean? • We have sampled more records than we theoretically need (but we’re still a bit paranoid, it’s early days with the survey) • Despite this, there are a larger number of strata that are samples rather than a census in 2014 (due to the retirement of the 1,333 rule) • Quotas will be set at 90%, +/-7.5% but we are aiming for 5%

  16. These workings are from SRC. If you would like access to the full version of this excel file, please contact SRC.

  17. Research framework & data collection

  18. What’s ‘the same’? • HEIMS provides the sample frame • Surveying at the level of the course, not the student • 100% online data collection • Letters used as the primary non-response follow-up strategy • Incentives and prize draw are unchanged • Live reporting of data collection Footer Text

  19. What’s new (questionnaire pt 1) ? • Using a standard rather than a rotated presentation of the questionnaire modules, • Removal of the Graduate Qualities Scale (GQS) and the Learning Community Scale (LCS) from the CEQ • Presentation of the CEQ scales to a sample of students in their final years across all institutions rather than undertaking a census of all students at a selection of universities (n=400 completed surveys)

  20. What’s new (questionnaire pt 2)? • Removal of the Student Support focus area item ‘At university during (year x), to what extent have you used university services to support your study?’, and • Adding two additional items at the conclusion of the UES confirming graduation intentions and collecting a private email address to facilitate interfacing the UES with the 14/15 AGS.

  21. What’s new (fieldwork)? • Option to use SMS instead of or as a supplement to the email reminder

  22. Live Reporting Module

  23. The Basics - Access • Your institution will be given a username and password just prior to your field date commencement to access your admin page: Example: http://survey1.mbdata.com.au/MonashDemo/admin.html username = admin password = jpcNHb Please note: usernames and passwords are case sensitive

  24. The Basics – Summary Tab • Your summary page will provide a quick overview of how the project is tracking

  25. Live Data – Record Search • This dialog allows you to search for specific records by different variables, or combinations of variables. For example, you could search for all records where Q1=5 (Variable Name "Q1", Operator "=", Value "5").

  26. Live Data - Data reports • The Data Reports area lets you run… • frequency tabulations • cross-tabulations • charts • …of the data in real-time. • Note: open-ended responses cannot be extracted with this function. Rows (stubs): the question you are tabulating. Columns(banners): cross-tab variables here. Filter by complete, partial or all respondents Run the tabulation. You can save a report once it’s created

  27. Live Data – Download raw data set • Open-ended responses can be accessed with this function. • Downloading data provides quick and easy access to the raw data set in excel format.

  28. Important points: • You’re dealing with live unedited data. • Downloading data and report outputs will be cumulative (i.e., Day 1, 2 & 3’s data will be in the output on Day 3, not just Day 3’s). • Getting too excited and creating lots of reports or downloading data during the first few days of launch can affect the performance of the survey. • You will receive a guide to the Admin Module

  29. Additional populations & items

  30. Additional populations • Enabling • Middle years • Offshore • Post graduate • Subsampling international students • Subsampling coursework students • No requests for benchmarking?

  31. Additional populations & fieldwork • Additional populations are removed from the monitoring fields in the reporting module • They are assigned an out of range subject area • The non-response follow-up strategy is tailored to each institution • Institutions that elect not to send a letter are aware of the 5% to 10% reduction in response rate • This data is not shared with the department or distributed in the National UES file Footer Text

  32. Additional items • Workplace Relevance Scale – six institutions • Appropriate Workload Scale – one institution • Net Promoter Score – two institutions • ‘how likely is it that you would recommend university x to friends/family/colleagues?’ • Follow up for potential leavers – three institutions • ‘you mentioned you were considering leaving. Would it be OK if someone contacted you to discuss options / what support you might need?’ • Institution specific – seven institutions Footer Text

  33. Additional items & fieldwork • All additional items are located in Module 5 before the CEQ except… • For those institutions that have added or reinstated a CEQ scale • CEQ scales over and above the ‘core scales’ are contained in Module 6 in the correct order for the administration of the CEQ Footer Text

More Related