1 / 25

Michael Lamport Commons Harvard Medical School Eva Yuja Li Dare Institute

Do The Ordinal Orders of Hierarchical Complexity Produce Significant Gaps Between Stages and Are the Stages Equally Spaced?. Michael Lamport Commons Harvard Medical School Eva Yuja Li Dare Institute. Presented at Piaget Society, Saturday, 9:00-10:30, June 2, 2012

mikasi
Télécharger la présentation

Michael Lamport Commons Harvard Medical School Eva Yuja Li Dare Institute

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Do The Ordinal Orders of Hierarchical Complexity Produce Significant Gaps Between Stages and Are the Stages Equally Spaced? Michael Lamport Commons Harvard Medical School Eva Yuja Li Dare Institute Presented at Piaget Society, Saturday, 9:00-10:30, June 2, 2012 SY21 Symposium Session 21, Symposium: Theory And Application Generated By The Model Of Hierarchical Complexity Conference Room B, Sheraton Centre, Toronto, CA

  2. Background • Model of Hierarchical Complexity is applied to many domains to measure the complexity of tasks • Tasks are assigned integer numbers called Orders of Hierarchical Complexity • As a measurement scale, its property is investigated in this paper • Is the Order of Hierarchical Complexity an ordinal scale? • Only an ordinal scale is meaningful • Are orders of hierarchical complexity linear and equally spaced? • A linear and equally spaced scale would indicate that there is equal amount of difficulty to move from one stage to the next

  3. Background • Is the Order of Hierarchical Complexity an ordinal scale? • As shown in the Rasch Variable Map, the stages of performance of items followed the same sequence of their orders of Hierarchical Complexity • In addition, ordinality might show up as gaps been the stages of performance on those items • Are orders of hierarchical complexity linear and equally spaced? • Equally spaced orders would indicate that moving from one stage to the next is always the same difficulty • If the performance measured by Rasch Analysis on the items are equally spaced, then the orders of Hierarchical Complexity would also be equally spaced

  4. Method Instrument • This study used the laundry instrument that was based on the Inhelder and Piaget’s (1958) pendulum task. • 111 items ranging from Primary Order 7 to Systematic Order 11 Procedure • Instrument was presented in a survey online • The tasks were presented in a sequence from easy to hard • The items were coded as correct or incorrect with missing answers being assumed incorrect • Data was analyzed using the Rasch Model

  5. Method • Participants: • 113 participants • 47 (41.6%) men and 66 (58.4%) women • Age18 to 100 (M = 35.8, (S.D. = 16.1). • Education • 35 high school graduates • 57 Bachelor’s degree holders • 8 master’s level degree holders and • 13 doctoral level degree holders • M = Bachelors degree

  6. Results • Rasch Analysis yielded two scales • Person Stage of Performance • Stands for how well the person performs on the set of tasks • Based solely on whether or not a given order of hierarchical Complexity is correctly carried out • Rasch Scaled Item Difficulty • How difficult items were empirically • This is the focus of this study

  7. Rasch Variable Map • Rasch Scaled Item Difficulty was ranged from -4.56 (Primary 7) to 3.94 (Systematic 11) • The higher an item is on the scale, the more difficult it is • P-Primary 7, C - Concrete 8, A- Abstract 9, F - Formal 10, S - systematic 11 • Rasch Scaled Item Difficulty of items sequenced in the same order as their Orders of Hierarchical Complexity • No item was out of order

  8. Are there “gaps” between Orders of Hierarchical Complexity? • Gaps are defined as “jumps” in Rasch Scaled Item Difficulty from one order to the next • Gaps may indicate that demands of tasks between adjacent orders have significant difference • Every other order adds a level of coordination • The order demands are supposed to be qualitatively different • Hypothesis: “Gaps” are significantly larger than the difference of Rasch Scaled Item Difficulty between items within each order

  9. Model to Test for Gaps • Let i= the observation number, which goes from 1 to 102. • DRi= β+ a7I 7i + a8I8i + a9I9i + a10I10i + a11I11i + εiwhere • DRi= the difference of Rasch Scaled Item Difficulty between item I and item (i-1) • β= the average of Gaps • an= the difference between the average of Item Break at order n and the average of Gap β • Ini= {1,0} {is, is not} a difference in Rasch scores for Hierarchical order or groupn • εiis a random variable fulfilling the Gauss Markov conditions

  10. Result • DR = 0.65500 - 0.57447 I7i - 0.58864 I8i -0.60553 I9i - 0.62237 I10i - 0.58397 I11i • This equation shows that the average of Gaps was 0.655. The average item break at each stage was smaller than the average gap size as shown by the an being negative. • There are 5 null hypotheses: an= 0, n = 7, 8, 9, 10, and 11 • There are 5 alternative hypotheses: an≠ 0, n = 7, 8, 9, 10, 11.

  11. Result • Five t-tests wereconducted • t7(97) = -10.014, p < 2-16 ≈ 0.00000 • t8 (97) = -9.667, p < 2-16 ≈ 0.00000 • t9 (97) = -10.555, p < 2-16 ≈ 0.00000 • t10 (97) = -10.848, p < 2-16 ≈ 0.00000 • t11 (97) = -10.499, p <2-16 ≈ 0.00000 • All the null hypotheses were rejected • Average Item Breaks are significantly smaller than the average Gaps • Therefore, we have shown that Gaps exist

  12. Test for Linearity and Equal Spacing • This section investigates whether the Order of Hierarchical Complexity was a linear and equally spaced scale • Four models were used • Simple regression model • Lack of fit test • Model on the spacing between Rasch Scaled Item Difficulty • Perturb the linear Order of Hierarchical Complexity

  13. Test for LinearitySimple linear regression • Simple linear regression between Rasch Scaled Item Difficulty y and Order of Hierarchical Complexity x • y= a + b*x • x = 7, 8, 9, 10, 11 - a linear scale • r = .983, r2 = .975 • The result shows that Item Order of Hierarchical Complexity predicts Rasch Scaled Item Difficulty with r of .983

  14. Test for LinearitySimple linear regression How to know whether a linear relationship is the best option to describe the data? • By comparing the variance explained by the linear regression model to variance explained by another model. The model that explains more variance is better • A Lack of Fit test compares the Linear Regression Model with the Separate Means Model • H0: Linear Regression Model explains significantly less variance than the Separate Means Model • H1:Linear Regression Model and the Separate Means Model explains equal amount of variance in the data

  15. Test for LinearitySimple linear regression • The lack of fit test shows that F(3) = 1.944, p = 0.128 • The separate means model does not explain significantly more variance than the linear regression model • The null hypothesis is that the spacing is unequal is not rejected • The result indicates that the linear relationship between the Task Order of Hierarchical Complexity and the Rasch Scaled Item Difficulty is not rejected by this analysis • The linearity assumption can still be held

  16. Test for Equal Spacing • Using a t-test, this analysis tests whether there are equal spacing between adjacent Orders of Hierarchical Complexity • Spacingis defined as the increment from the average of Rasch Scaled Item Difficulties of a lower order to the average of Rasch Scaled Item Difficulty of the next higher order • There are four spacings as there are five Orders of Hierarchical Complexity • A statistical Model is constructed to account for the differences of Rasch Scaled Item Difficulty between items

  17. Test for Equal Spacing • RD = RaschScaled Item Difficulty =β7+ γ8I8i+ γ9I9i+ γ10I10i+ γ11I11i+εi • RD= Rasch Scaled Item Difficulty; • Ini= {1, 0} when the item {is, is not} at the Order of Hierarchical Complexity denoted by n.n= {7, 8, 9, 10, 11}; • β7= is the average value of the Rasch Scaled Item Difficulty for items in order 7 • γ8 = the estimate of the difference between the average Rasch Scaled Item Difficulty at order 8 score and average Rasch Scaled Item Difficulty at order 7 score

  18. Test for Equal Spacing • β7 + γ8 estimates the average Rasch Scaled Item Difficulty at order 8 • {β7 + γ9, β7 + γ10, β7 + γ11} estimates the average Rasch Scaled Item Difficulty at order 9, 10 and 11 • H01: The spacing between order 9 and 8 is the same as the spacing between order 8 and 7. Or γ9 - 2γ8 = 0. • H02: The spacing between order 10 and 9 is the same as the spacing between order 9 and 8. Or, γ10 - 2 γ9 + γ8 = 0 • H03: The spacing between order 11 and 10 is the same as the spacing between order 10 and 9. Or, γ11 - 2γ10 + γ9 = 0 • One sample t-tests were used to test these hypotheses

  19. Test for Equal Spacing • The result shows that we cannot reject any of these null hypotheses: • For H01, t(97) = 0.240, p= 0.595 • For H02, t(97) = 0.0526, p= 0.479 • For H03, t(97) = 0.7949, p= 0.214 • Therefore, we cannot reject the null hypotheses that all the spacing between the orders is the same. This result is consistent the result of lack of fit test, which cannot reject linearity of the Orders of Hierarchical Complexity.

  20. Test for Equal SpacingPerturbation Test • Tests above supported that Order of Hierarchical Complexity as linear and equally spaced scale • However, it was due to the lack of evidence to reject null hypotheses, which does not prove the alternative hypotheses • This section of the paper takes an alternative route • We add noise to the Orders of Hierarchical Complexity scale • We test how much noise added to the scale would reject the linearity hypothesis • It will show the upper limit to the deviation from a linear scale

  21. Perturbation TestProcedure 1. Take the Orders of Hierarchical Complexity {7, 8, 9, 10, 11} 2. Randomly add or subtract 0.05 (randomly selected by computer) 3. Run a linear regression of the Rasch Scaled Item Difficulty on the newly defined order scale, obtain r of the model 4. Repeat step 2 three more times 5. Average four r’s, obtain the average r when OHC was perturbed with noise of 0.05 6. Repeat step 1-5 with noise level 0.1, 0.15, 0.2, … 0.45 • Stop at 0.45 because noise of 0.5 may subvert the sequence of OHC, thus violate ordinality of the scale

  22. Perturbation TestResult • This scatter plot is the size for perturbation versus predictability r • It shows that as the scale deviates from the original linear scale, predictability decreases steadily

  23. Perturbation TestResult • Using the Fisher r-to-z transformation, the significance of the ther found in the original linear regression model and the r's found in the new models when the Order of Hierarchical Complexity is perturbed were assessed • It is found that perturbing the Order of Hierarchical Complexity by more than 0.25 produces a significant difference in the predictability of the scale • When the noise was 0.25, the difference was significant at the 0.1 level, with z = 1.68, p = 0.093. When noise = 0.35, the difference was significant at 0.05 level, with z = 2.74, p = 0.006.

  24. Summary • Rasch Analysis showed that the Order of Hierarchical Complexity is an ordinal scale, where Orders predicted the relative difficulty of items • Simple linear regression showed that Orders of Hierarchical Complexity predicted Rasch Scaled Item Difficulty with an r of • lack of fit test showed that the linearity of the scale could not be rejected • Testing for Equal Spacing showed that the equal spacing assumption could not be rejected • Perturbing the scale by 0.25 led to a significant difference in the predictability of the scale

  25. Discussion • The existence of gaps shows that the ordinal nature of the scale is not just an assumption • MHC is an equally spaced ordinal scale • It is not an interval scale because it does not have additively or any cancellation property • Equal Spacing indicates that going from one order to the next produces equal difficulty between stages • This allows one to treat orders as real numbers, and not just indication of relative position • It might mean the order of Hierarchical Complexity, n, is a measure the quantity of hierarchical information • The minimum number of order ntask may be 2n • Given that tasks at order n+ 1 are defined by and coordinate 2 or more tasks at order n

More Related