ࡱ> _^O#(p4 / 00DArialBlacx(0(B 0 DTimes New Roman(0(B 0  DWingdingsRoman(0(B 0 0DArial Blackman(0(B 0 "@ .  @n?" dd@  @@`` @'         !"#$% 0AA@8wʚ;ʚ;g4AdAd@B 0ppp@ <4ddddlpC 0x <4BdBdlhG 080___PPT10 ?  %OKAIRP Fall Conference#NSU @ Broken Arrow November 4, 2005SDesigning Methods for Determining Educational Gain: Involving Faculty in AssessmentTT&?Mark L. Giese Northeastern State University Tahlequah, Oklahoma)Measuring Program Dispositions as a Model**&Knowledge Skills DispositionsWhat are dispositions?What is the big deal about dispositions? Dispositions are: Concepts we inherently value as a profession. Esoteric to the discipline.&<J<JWhat are dispositions in law?(Ethical Passion for service Accessible#What are dispositions in Education?$$&zIn Health and Kinesiology, dispositions may include: Hard work Teamwork Our students or clients can improve with practice6E{Who is involved?QUse all persons with a stake: Faculty Administrators Let faculty make mistakes Why measure?Looks at the Affective Learning Domain. In the past, typical areas of measurement have only included knowledge and skills. Accrediting agencies (learned societies) require measurement in this domain. 4Steps in developing a measurement tool for your unit55&Planning Review the literature and ask your friends what they are doing. Decide on which dispositions to measure (committee). Decide if you intend to use this exercise as an exit competency or if you are looking for a change from entry to exit. Determine an a priori standard.:    4Steps in developing a measurement tool for your unit55&Implementation Devise a measurement tool (rubric) Collect your data Analyze your data Determine if an intervention is necessary REMEASURE!&|| ExamplesuExample of a college rubric for measuring dispositions in pre-service teacher education candidates in the COE at NSU. ExamplesExample of a Sociology department rubric for measuring dispositions: Attempt to measure  Social Distance (F&s   ImportanceMeasuring dispositions is important because it gives us another piece of information to determine our effectiveness and quality. ,Examples of Presentations on Social Distance--& Social Distance as a Disposition of Teacher Candidates. Presented at AACTE, Chicago, Illinois, February 8, 2004. ,Examples of Presentations on Social Distance--&6 Attitudes Towards Social Distance of Pre-Service Teacher Education Students. Presented at Higher Education Research Day, Edmond, Oklahoma, November 2003. ,Examples of Presentations on Social Distance--& Attitudes Towards Social Distance of Pre-Service Teacher Education Students. Presented at the Oklahoma Association of Colleges for Teacher Education Conference, Norman, Oklahoma, November 9, 2000.Two Types of Measurement &=One-time (barrier) Demonstrate program effectiveness (gain)()>Measurement GoalsOne Time: Graduation requirement etc. Using a pre/post test design in an attempt to determine program gain. May be done at entry and exit of any major.Does your data fit the model?%Parametric (t, ANOVA) Non-parametricDetermining Program GainMeasure at pretest. Measure at post test. Determine the difference. Use hypothesis testing to determine if a significant difference exists. How ANCOVA worksThe variable under study is the dependent variable and the covariate is the  other data point. Both must be known for each subject (matched).!How ANCOVA worksThe influence of the covariate is removed using a straight-forward linear regression method. The ANCOVA produces an adjusted post test score for each subject and an adjusted mean as well.ANCOVA ExampleThree Methods of teaching French: Pretest& & ..instruction& & Post test Measures of intelligence available Remove the effect of intelligence so the French method may be the cause of the change from pre/post score.ANCOVA>Group 1 Pre Post . . . . . . . . . . . . . . . . X X0? .Assumptions of ANOVAMHomogenity of variance Equal distribution Random sampling Additive effectsExtra Assumptions of ANCOVAlIV does not influence the covariate variable Equal regression slopes Linearity r of .60 (Shavelson, 1996)mZm\ ReviewAsk faculty if they need/desire to measure dispositions. Assist with choosing an instrument and measurement model. Assist in determining gain and the magnitude of the effect. Questions ??QUESTIONS??  ` fffff` ?̙ffE` ff333` C3ff3` &u3` ffff` **@fff̙fl` +3[Xd` 333f` 5D`|f>?" dd@*?lPd@ `F `A@`<``( n?" dd@   @@``PP   @ ` `.p>> @ T(  2T ```  "```"  BDxG p"P`` D0 B  C "H0 D0 B  C  ~BCE6FGIQSTUV.WX@` B C "h  B0 B  s *Dp"p,$ 0  0< "py  T Click to edit Master title style! !  0 "x   W#Click to edit Master subtitle style$ $   60 #" `` `  X*    6 = #" `c   Z*    6A #" `` `  \* H  0޽h ? fffff80___PPT10. p |(  |  | Ngr4gr4 )   v* U++UU | Nԩgr4gr4  o)  x* U++UU | TJgr4gr4  3   v* U++UU | TtEgr4gr4   o3  x* U++UUH | 0:l ? 3380___PPT10.@o  0(  x  c $Lpy  x  c $Lx   H  0޽h ? 33___PPT10i.!+D=' = @B + p<(  ~  s *py  ~  s *xx   H  0޽h ? 33___PPT10i.!+D=' = @B +}  p`$(  `r ` S |"{4   r ` S P#  H ` 0޽h ? fffff___PPT10i."+D=' = @B +}  `$(  r  S .{4   r  S /  H  0޽h ? fffff___PPT10i.!+D=' = @B +}  d$(  dr d S 5{4   r d S 5  H d 0޽h ? fffff___PPT10i.@ +D=' = @B +}   $(   r  S X{4   r  S @  H  0޽h ? fffff___PPT10i."IU+D=' = @B +}  h$(  hr h S {4   r h S   H h 0޽h ? fffff___PPT10i.`++D=' = @B +}  $$(  $r $ S 0{4   r $ S   H $ 0޽h ? fffff___PPT10i."++D=' = @B +}  ($(  (r ( S {4   r ( S h  H ( 0޽h ? fffff___PPT10i."w|M+D=' = @B +}  ,$(  ,r , S {4   r , S   H , 0޽h ? fffff___PPT10i."|+D=' = @B +}  0$(  0r 0 S {4   r 0 S 4  H 0 0޽h ? fffff___PPT10i." `I+D=' = @B +}  4$(  4r 4 S {4   r 4 S \  H 4 0޽h ? fffff___PPT10i.#w+D=' = @B +}  8$(  8r 8 S G{4   r 8 S 4H  H 8 0޽h ? fffff___PPT10i.#^+D=' = @B +}  <$(  <r < S /{4   r < S T0  H < 0޽h ? fffff___PPT10i.%+D=' = @B +  D0(  Dx D c $8{4   x D c $9  H D 0޽h ? fffff___PPT10i.%+D=' = @B +  0L0(  Lx L c $lB{4   x L c $C  H L 0޽h ? fffff___PPT10i.%+D=' = @B +}  @P$(  Pr P S  {4   r P S   H P 0޽h ? fffff___PPT10i.%H+D=' = @B +}  $(  r  S (%{4   r  S ,  H  0޽h ? fffff___PPT10i.Lp +D=' = @B +}  PT$(  Tr T S `{4   r T S    H T 0޽h ? fffff___PPT10i.%r+D=' = @B +}  $(  r  S {4   r  S   H  0޽h ? fffff___PPT10i.O^+D=' = @B +}  0$(  r  S Ll{4   r  S l  H  0޽h ? fffff___PPT10i. +D=' = @B +}  @$(  r  S lO{4   r  S @P  H  0޽h ? fffff___PPT10i.z3+D=' = @B +}   $(  r  S U{4   r  S xX  H  0޽h ? fffff___PPT10i.<+D=' = @B +Y  ph` X(  Xr X S ]{4   r X S H  LB X c $D ` LB X c $D   X 0'   >Group 2 Pre Post . . . . . . . . . . . . . . . . X X0? .LB X c $D0 @ 0 LB X c $D0 @ 0   X 0^ P >Group 3 Pre Post . . . . . . . . . . . . . . . . X X0? .LB  X c $D( h( LB  X c $D( ( H X 0޽h ? fffff___PPT10i.%put+D=' = @B +}  x$(  xr x S p{4   r x S q  H x 0޽h ? fffff___PPT10i.A+D=' = @B +}  l$(  lr l S x{4   r l S y  H l 0޽h ? fffff___PPT10i.U+D=' = @B +}  p$(  pr p S Ȇ{4   r p S   H p 0޽h ? fffff___PPT10i. +D=' = @B +}  t$(  tr t S {{4   r t S   H t 0޽h ? fffff___PPT10i.+D=' = @B +r0W#0@C?@ABCDEGHIJKLMOPQRSTUWXYZ[\]`Root EntrydO)Current UserVSummaryInformation(FPowerPoint Document(DocumentSummaryInformation8N