Wednesday, December 14, 2016

Data Analysis Tools Week4 - Moderating Variables

Way ahead of schedule

Since I had a little extra time on my hands, I've been able to dedicate a lot of my time to working through these assignments. I'm thankful for that.

Moderating variable

I've shown an association between grade retention in school and the propensity for violence in previous assignments. Today, I'm looking at the age a respondent first starts drinking alcoholic beverages as a possible moderating variable. I took the data and broke it down into four categories, Age 0-6, Elementary, Secondary, and Never.

I looked at each of the variables and determined that only Never drinking or beginning to drink at Secondary levels has any effect on violence among students have been retained.

Preschool drinking

  1. sub1=clean[(clean['FIRSTAGE']=='Preschool')]  
  2. ctPS=pd.crosstab(sub1['VIOLENT'], sub1['RETAINED'])  
  3. print("\nChi-Square of Retention and Violence among students who first drank\  
  4. alcohol before age 6")  
  5. print(ctPS)  
  6. print("\n\nChi-square value, p-value, expected counts")  
  7. xsquare1=sps.chi2_contingency(ctPS)  
  8. print(xsquare1)  
  9.   
  10. seaborn.factorplot(x="RETAINED", y="VIOLENT", data=sub1, kind="bar", ci=None)  
  11. plt.xlabel('Retained at least one grade')  
  12. plt.ylabel('Percentage Violent')  
  13. plt.title("Association of retention and violence among respondents who \  
  14. first drank alcohol before age 6")  

Chi-Square of Retention and Violence among students who first drankalcohol before age 6
RETAINED  No  Yes
VIOLENT          
0          4    0
1          6    3


Chi-square value, p-value, expected counts
(0.36412037037037032, 0.54622674479725641, 1, array([[ 3.07692308,  0.92307692],
       [ 6.92307692,  2.07692308]]))

The Chi-Square is low and the p-value is far too high. The null hypothesis would be accepted in this case.

Drinking in Elementary School

  1. sub2=clean[(clean['FIRSTAGE']=='Elementary')]  
  2. seaborn.factorplot(x="RETAINED", y="VIOLENT", data=sub2, kind="bar", ci=None)  
  3. plt.xlabel('Retained at least one grade')  
  4. plt.ylabel('Percentage Violent')  
  5. plt.title("Association of retention and violence among respondents who \  
  6. first drank alcohol 6-12")  
  7.   
  8. ctElem=pd.crosstab(sub2['VIOLENT'], sub2['RETAINED'])  
  9. print("\nChi-Square of Retention and Violence among students who first drank \  
  10. alcohol ages 6-12")  
  11. print(ctElem)  
  12. print("\n\nChi-square value, p-value, expected counts")  
  13. xsquare2=sps.chi2_contingency(ctElem)  
  14. print(xsquare2)  
Chi-Square of Retention and Violence among students who first drank alcohol ages 6-12
RETAINED   No  Yes
VIOLENT           
0          94   38
1         107   53


Chi-square value, p-value, expected counts
(0.44815984181655855, 0.50321011393827875, 1, array([[  90.8630137,   41.1369863],
       [ 110.1369863,   49.8630137]]))

The Chi-Square is low again and the p-value is again too high. The null hypothesis is accepted in this case as well.

Drinking in Middle and High School


  1. sub3=clean[(clean['FIRSTAGE']=='Secondary')]  
  2. seaborn.factorplot(x="RETAINED", y="VIOLENT", data=sub3, kind="bar", ci=None)  
  3. plt.xlabel('Retained at least one grade')  
  4. plt.ylabel('Percentage Violent')  
  5. plt.title("Association of retention and violence among respondents who \  
  6. first drank alcohol 12-18")  
  7.   
  8. ctSec=pd.crosstab(sub3['VIOLENT'], sub3['RETAINED'])  
  9. print("\nChi-Square of Retention and Violence among students who first drank \  
  10. alcohol ages 12-18")  
  11. print(ctSec)  
  12. print("\n\nChi-square value, p-value, expected counts")  
  13. xsquare3=sps.chi2_contingency(ctSec)  
  14. print(xsquare3)  
Chi-Square of Retention and Violence among students who first drank alcohol ages 12-18
RETAINED   No  Yes
VIOLENT           
0         470  111
1         252  132


Chi-square value, p-value, expected counts
(27.808663811593675, 1.3392503595490095e-07, 1, array([[ 434.69637306,  146.30362694],
       [ 287.30362694,   96.69637306]]))

The null hypothesis is rejected for students who begin drinking at the secondary level. The Chi-Square is 27.8 and the p-value is very low.

Never drank alcohol

  1. sub4=clean[(clean['FIRSTAGE']=='Never')]  
  2. seaborn.factorplot(x="RETAINED", y="VIOLENT", data=sub4, kind="bar", ci=None,  
  3.                    x_order=['Yes', 'No'])  
  4. plt.xlabel('Retained at least one grade')  
  5. plt.ylabel('Percentage Violent')  
  6. plt.title("Association of retention and violence among respondents who \  
  7. never drank alcohol")  
  8.   
  9. ctnever=pd.crosstab(sub4['VIOLENT'], sub4['RETAINED'])  
  10. print("\nChi-Square of Retention and Violence among students who never drank \  
  11. alcohol")  
  12. print(ctnever)  
  13. print("\n\nChi-square value, p-value, expected counts")  
  14.   
  15. xsquare4=sps.chi2_contingency(ctnever)  
  16. print(xsquare4)  

Chi-Square of Retention and Violence among students who never drank alcohol
RETAINED    No  Yes
VIOLENT            
0         3034  626
1         1119  427


Chi-square value, p-value, expected counts
(73.838722844073715, 8.4767300358856875e-18, 1, array([[ 2919.70418748,   740.29581252],
       [ 1233.29581252,   312.70418748]]))
This final section showed the highest Chi-Square and the lowest p-value. The null hypothesis is rejected for this group.

The whole code as it was written

  1. import pandas as pd  
  2. import numpy as np  
  3. import scipy.stats as sps  
  4. import seaborn  
  5. import matplotlib.pyplot as plt  
  6.   
  7. filename="addhealth_pds.csv"  
  8. data = pd.read_csv(filename, low_memory=False)  
  9.   
  10. data['H1ED5']=data['H1ED5'].replace(6, np.nan)  
  11. data['H1ED5']=data['H1ED5'].replace(8, np.nan)  
  12. data['H1TO4']=data['H1TO4'].replace(96, np.nan)  
  13. data['H1TO4']=data['H1TO4'].replace(98, np.nan)  
  14.   
  15. def violent (row):  
  16.     if row["H1FV5"] >= 1 and row["H1FV5"] <= 2:  
  17.         return 1  
  18.     if row["H1FV7"] >= 1 and row["H1FV7"] <= 2:  
  19.         return 1  
  20.     if row["H1FV8"] >= 1 and row["H1FV8"] <= 2:  
  21.         return 1  
  22.     else:  
  23.         return 0  
  24. data['VIOLENT'] = data.apply(lambda row: violent (row), axis = 1)  
  25.   
  26. drinkage=('Preschool', 'Elementary', 'Secondary', 'Never')  
  27. data['FIRSTAGE']=pd.cut(data['H1TO4'],(0, 6, 12, 18,100), labels=drinkage)  
  28.   
  29. data['RETAINED']=data['H1ED5']  
  30.   
  31. clean=data[['RETAINED', 'FIRSTAGE', 'VIOLENT']].dropna()  
  32. clean['RETAINED']=clean['RETAINED'].astype(int)  
  33. remap1={1:'Yes', 0:'No',}  
  34. clean['RETAINED']=clean['RETAINED'].map(remap1)  
  35.   
  36. countlevels=clean['FIRSTAGE'].value_counts()  
  37. print(countlevels)  
  38.   
  39. sub1=clean[(clean['FIRSTAGE']=='Preschool')]  
  40. ctPS=pd.crosstab(sub1['VIOLENT'], sub1['RETAINED'])  
  41. print("\nChi-Square of Retention and Violence among students who first drank\  
  42. alcohol before age 6")  
  43. print(ctPS)  
  44. print("\n\nChi-square value, p-value, expected counts")  
  45. xsquare1=sps.chi2_contingency(ctPS)  
  46. print(xsquare1)  
  47.   
  48. seaborn.factorplot(x="RETAINED", y="VIOLENT", data=sub1, kind="bar", ci=None)  
  49. plt.xlabel('Retained at least one grade')  
  50. plt.ylabel('Percentage Violent')  
  51. plt.title("Association of retention and violence among respondents who \  
  52. first drank alcohol before age 6")  
  53.   
  54. sub2=clean[(clean['FIRSTAGE']=='Elementary')]  
  55. seaborn.factorplot(x="RETAINED", y="VIOLENT", data=sub2, kind="bar", ci=None)  
  56. plt.xlabel('Retained at least one grade')  
  57. plt.ylabel('Percentage Violent')  
  58. plt.title("Association of retention and violence among respondents who \  
  59. first drank alcohol 6-12")  
  60.   
  61. ctElem=pd.crosstab(sub2['VIOLENT'], sub2['RETAINED'])  
  62. print("\nChi-Square of Retention and Violence among students who first drank \  
  63. alcohol ages 6-12")  
  64. print(ctElem)  
  65. print("\n\nChi-square value, p-value, expected counts")  
  66. xsquare2=sps.chi2_contingency(ctElem)  
  67. print(xsquare2)  
  68.   
  69. sub3=clean[(clean['FIRSTAGE']=='Secondary')]  
  70. seaborn.factorplot(x="RETAINED", y="VIOLENT", data=sub3, kind="bar", ci=None)  
  71. plt.xlabel('Retained at least one grade')  
  72. plt.ylabel('Percentage Violent')  
  73. plt.title("Association of retention and violence among respondents who \  
  74. first drank alcohol 12-18")  
  75.   
  76. ctSec=pd.crosstab(sub3['VIOLENT'], sub3['RETAINED'])  
  77. print("\nChi-Square of Retention and Violence among students who first drank \  
  78. alcohol ages 12-18")  
  79. print(ctSec)  
  80. print("\n\nChi-square value, p-value, expected counts")  
  81. xsquare3=sps.chi2_contingency(ctSec)  
  82. print(xsquare3)  
  83.   
  84. sub4=clean[(clean['FIRSTAGE']=='Never')]  
  85. seaborn.factorplot(x="RETAINED", y="VIOLENT", data=sub4, kind="bar", ci=None,  
  86.                    x_order=['Yes', 'No'])  
  87. plt.xlabel('Retained at least one grade')  
  88. plt.ylabel('Percentage Violent')  
  89. plt.title("Association of retention and violence among respondents who \  
  90. never drank alcohol")  
  91.   
  92. ctnever=pd.crosstab(sub4['VIOLENT'], sub4['RETAINED'])  
  93. print("\nChi-Square of Retention and Violence among students who never drank \  
  94. alcohol")  
  95. print(ctnever)  
  96. print("\n\nChi-square value, p-value, expected counts")  
  97.   
  98. xsquare4=sps.chi2_contingency(ctnever)  
  99. print(xsquare4)  
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)