Do More Freshmen Surveys Lead to More Freshmen Involved?

June 6, 2013 — 10 Comments

On my Stuff You Can Use for the First Week on Campus post (which has spiritual interest surveys, fliers, brochures, and other free & helpful stuff on it), a recent commenter, Ron Cram, wanted more details on how we do spiritual interest surveys. And since I love data analysis I couldn’t resist sharing what the data tells us.

I think it’s pretty interesting to see what correlation there is between

  • number of surveys done the first week of school AND
  • number of freshmen that actually get involved (in Bible studies)

 

So here’s Ron’s comments/questions and my answers:

Tim, I am interested in an analysis of the data on the card. It sounds like you got 3000 students to complete the card…

We do about 3000 surveys over the first few weeks at various events. The stats below reflect the 2500 surveys we do at tables we set up outside of dorm cafeterias on the second and third day of class. Why not the first day? Because our Cru meeting is on Tuesday, and Dorm Studies on Wednesday. When we do a survey with them we give them a “Free Chick-fil-A Sandwich” card and a flyer for Cru and Dorm Bible Studies (and we say, “you should join us tonight at Cru/Bible Study”)

How many of them indicated an interest in Cru?

1 Minute Questionnaire

Click to see full size

To download a Photoshop file you can edit to use on your campus, click here.

We don’t keep stats specifically on each question because it doesn’t affect how we do follow up. We follow up anyone who checks “yes” on either question.

70% of students checked “yes” on one of the two questions.

30% of the respondents answered “no-no”. Not interested in Cru nor Bible studies (we don’t follow them up at all).

Here’s the breakdown of how they answered:

“How interested are you in exploring spiritual matters in college (1- not interested 5= very interested)?”

1 = 7%
2 = 13%
3 = 29%
4 = 21%
5 = 24%
No answer = 6%

So on our campus, about 25% of students are not interested in spiritual things. 75% are at least mildly interested. How does that compare to your campus?

How many actually got involved?

We noticed this a few years ago: it’s not important how many spiritual surveys we do, as long as we’re doing enough to have a plenty big pool of contacts to follow up (I’d say around 1000-1500). Doing more surveys does not result in more freshmen getting involved (at least for us).

That being said, we do feel that it is good to do surveys with as many freshmen as possible for several reasons:

      • Every person we do a survey with, we get face to face with and invite them verbally and with a flier to a Cru event. And they get something free (Chick-fil-A card or sunglasses) – hopefully a very positive first experience with Cru.
      • If we can do surveys with a high percentage of the freshmen class, we have a baseline understanding of where A LOT of students are at spiritually. As we bump into students later in the year (or the next 4), we can quickly look them up on Mission Hub and know “Michael was not very interested in spiritual things at all in August 2012 – he put 1-yes-no. Joey Smith met with him and invited him to a Bible study but he never came” 

Here’s what our stats showed us:

surveys vs freshmen involved

You notice from 2008 to 2009 we doubled the number of surveys we did. But it has zero impact on getting more freshmen involved.

Here’s what matters and causes more freshmen to get involved:

    1. Having more student Community Group leaders. You can see how the growth in Freshmen in studies correlates with (and I would say is caused by):

      study leaders

    2. The quality and thoroughness of follow up

 

How many people (staff or students) were involved in collecting this data? How long did it take? Was it all done in one day?

We do tables for two days at five locations (4 dorm cafeterias and the Union) from 11am-1pm and 5-6:30pm.

Our 10 staff are at the tables both days at lunch. Students are present at the tables for both lunch and dinner. I would guess that we have 30 students at lunch and 30 at dinner each day. Maybe a total of 50-75 helping during the two days?

Right after we collect all the cards we divvy up the cards among students and they enter in the information into Mission Hub. I have no idea how long that takes. I would guess 5 hours for about 20 students?

We also do spiritual interest surveys at two big freshmen cookouts during move-in week, a midnight “Frisbees and Flapjacks” event, and our Cru meetings.

 

Hopefully that data/information is helpful for you as think through a gameplan for getting in contact with (and reaching!) freshmen in the Fall.

Would love to hear from you what you have seen on your campus – what has resulted in you getting more freshmen involved?

 

timcasteel

Posts Twitter

P.S. - I'd love to connect with you on Twitter: here
  • Andrew Wise

    Cool data. Is the Student Bible Study Leader number the number of leaders who are specifically focused on reaching freshmen, or just bible study leaders overall? If overall, do you have the specific breakdown of freshman-focused leaders, and is there still the correlation there? What’s the average number of freshmen that get involved as the result of an additional freshman bible study leader getting involved?

    • Anonymous

      I’m sure I could go back and count up freshmen leaders but, unfortunately we didn’t delineate that in our count over the years. I know this year we had 100 freshmen study leaders (out of 138). And I vaguely remember looking back and noticing that ratio is pretty true every year (72% of our leaders are going after freshmen).

      It looks like the avg # of freshmen that get involved per additional freshman bible study leader is all over the place.

      60 new leaders in 2012 resulted in 60 new freshmen involved. Not great.

      36 additional leaders in 2013 resulted in about 100 new fish.

      • Andrew Wise

        That’s all pretty understandable, I think, as it probably indicates that both quantity of new leaders pursuing freshmen and the quality of those leaders are variables that influence the outcome. Good stuff.

  • Ron Cram

    Tim, thank you for this! Very interesting data. I’m not a statistician and there are probably factors I am not aware of, but my first thought is both the number of surveys and the number of leaders play a role. Between 2008 and 2010, there was a 50% increase in number of leaders and no real growth in the number of freshmen. That data does not strongly support your hypothesis. This may be due, in part, to the quality of the leaders or possibly some other factors. Between 2010 and 2011, the number of leaders went down slightly but the number of surveys went up and you saw 10% growth in the number of freshmen. The data on the change between 2011 and 2012 is the strongest for your hypothesis. The number of surveys was unchanged but the number of leaders more than doubled and the number of freshman went up by 63%.

    The data you gathered on interest in spiritual matters may be misleading. Christians are likely to look at that data and assume the students are interested in Christianity. Not likely. In the surveys I have conducted with students, most of them describe themselves as “being on a spiritual journey” but these same students believe Christianity is not intellectually viable and is not compatible with science. They are not ready to give the gospel message a hearing. The spirituality they are interested in may be an eastern spirituality or an American Indian spirituality or perhaps they just want to seek their own way.

    I am strongly convinced the #1 reason college students are not very interested in Christ is the same reason why 70% of youth group kids walk away from the church during their college years – they believe science has defeated Christ and Christianity.

    God sent John the Baptist to prepare the way for Jesus. And Jesus said he was successful – “the fields are white unto harvest.” We cannot say that on the college campus. We need a fresh movement of cultural pre-evangelism, preparing the way by breaking up the hard soil so the gospel seed can be planted. We have to show that Christianity is intellectually viable or we will lose this generation. This is why I wrote my booklet.

    • Anonymous

      Thanks for taking the time to comment Ron. I agree – it’s not a completely cut and dry argument. But I do think there’s a stronger correlation between Study leaders and freshmen involved. And as I said, as long as you are doing ENOUGH surveys (around 1000-1500) then there’s not a huge gain from more.

      I do think the intellectual credibility of Christianity is AN issue. I’m just not convinced it’s THE issue. Or at least not in the way you frame it: “they believe science has defeated Christianity”. I definitely wouldn’t say it’s the #1 reason college students/people reject God.

      I think they find God to be irrelevant to their life and aspirations (see this post: http://www.timcasteel.com/2010/08/chief-end-of-students/ ).

      Or they’ve tried “christianity”/religion before and are now inoculated – http://www.timcasteel.com/2012/05/the-gospel-for-bible-belt-christians/

      Or they see us as morons. But not necessarily because of Science. But because we act like morons sometime. I’m using the royal we – meaning not me 🙂 “We”= Joel Osteen, Westboro Baptist, Pat Robertson, etc. They see all those people and think they speak for Christianity.

      Anyway – just my take on it. And definitely coming out of my context which is the Bible belt. I know your audience you’ve seen could be totally different.

      • Ron Cram

        Thank you for taking the time to comment. Many of your points are well-taken. I think there are ways to determine how much Scientism – the belief science has defeated the Bible and Christ – is affecting your local campus. What percentage of the student population at your campus are science majors (physics, chemistry, biology, pre-med, etc.)? Don’t include the social science majors in that count because the soft sciences are not real science. Next, what percentage of the students in your ministry are in those science majors? Is the percentage of science majors involved in Cru less than the campus as a whole? I’m guessing the answer is yes – even in the Bible belt.

        In my opinion, there is a trickle down effect. The science majors are likely to think that the material universe is all that exists and therefore all religious people are uneducated, bigoted and generally to be avoided. The non-science majors are more likely to be open to the idea life is spiritual but tend to pick up the attitudes from science majors and from the culture that Christians are uneducated, bigoted and generally to be avoided. Out here in California, a big part of those interested in spiritual things are not interested in Christianity. But if they think Christianity is compatible with science, then they are far more open to the gospel.

        We cannot escape the fact that our culture values scientific knowledge. And science has been remarkably effective in explaining the natural world, finding cures to disease and enabling the advance of technology and society. In our modern society, the epistemological question is no longer “How do you know?” but “What does the data say?”

        Another way to measure the impact of Scientism on your campus is survey the students. Have you seen the survey I use? I have modified it recently to include four more questions. Here are three of them:

        * Are science and Christianity compatible?

        * Is Christianity intellectually viable?

        * Do you consider yourself to be on a spiritual journey?

        Possible answers are: Yes – Leans Yes – No Opinion – Leans No – No.

        The vast majority will answer the first two questions as either “No” or “Leans No.” The vast majority will answer the last question “Yes” even if they are atheists (unless they are science majors). How can atheists believe life is spiritual? Because humans are full of inconsistencies. Culture says life is spiritual. Science says it is not. Double-think is very common on our campuses.

        Anyway, those are my thoughts based on the data I’ve collected.

        • Anonymous

          Very interesting Ron – your data you’ve gathered – especially re: double-think.

          Our movement is probably pretty representative of the entire campus. We tend to attract pretty intelligent/honor students. Many from engineering majors – I’m not sure how many are pure science majors. But that’s a result of years of success in the major honors dorm on campus. Anyway – I suspect California is far different from Arkansas but I think the beliefs represented in California are where our country is headed. So it’s good to get insight from folks like you on the front lines.
          Thanks for your thoughts.

  • James

    I was a little surprised to see that the surveys didn’t correlate to more freshmen, but that may be because some are turned off by the surveys and follow-ups. It may remind some students of the cults that continually pester students until they commit to different things. I know many horror stories about that and sadly so do many of the other students. It could also be that the card does not say anything about a follow-up, so many students may say that they have some interest but would not expect anything to be initiated, and would probably not expect any follow-up. When they do get the follow-up, that might turn them off because they may feel like they are being followed. I think having more student leaders who interact with everyone else on a daily basis is a much more effective means of ministry because it is not only less intrusive but also lasts a much longer period of time and gives them an example of Christ and not just a discussion about Him.

    As far as getting a more representative sample, as long as you can expect 10 freshmen to answer to each of the given categories (1, 2, 3, 4, 5, none), your sample, if taken randomly, should give a good representation of the entire freshman class. Your smallest category is about 6% of the total, so if 6% is 10 students, then the minimum total respondents should be at least 167 respondents. Anything above this just enhances the precision of your survey without really changing the big picture of your data. Just leaving the surveys at the dining commons (this being almost inclusive of the freshman pool) and expecting at least 250 respondents should honestly be enough, and this is absolutely regardless of whether or not there are 1000 freshman or 1 million, as long as nearly all of the freshman use the dining commons. I know that is the case at UMass Amherst, but it might be different at your college.

    I think what is also important in the data is the sudden influx of new leaders during 2010 and 2011. That gave a ton of new role models with whom the new students could more readily identify, and that would, in my opinion, explain the sudden surge in newcomers and new leaders in 2012.

  • Rahul Agarwal

    Tim, thanks for this, you are really helping me out a lot! Maybe I missed this but what is the purpose behind each question? What determines who you contact and who you don’t? Can you expand on that vs. just having the last question?

    • timcasteel

      Good questions.
      The questions are supposed to go from broad to narrow:
      “How interested are you in exploring spiritual matters in college?” – Gives us a gauge of their spiritual temperature

      Interest in info about how to grow spiritually is less threatening than Bible study.
      We pursue the “5, yes, yes” first. We never contact the “no, no”. We don’t really have a great system for how we prioritize the others.