Research Methods in Instructional Technology
Lloyd Rieber, Instructor
(Last updated on June 15, 2018 .)
This course introduces you to key concepts, practices, and skills in the educational research process. This context is organized into two major areas:
We will explore different aspects of the research experience during the term as we focus on the overall research process. In the first half of the course, you will participate in Research Design Activities (RDAs) for you to learn how to critically examine real world research. You will also learn some fundamentals of statistics in a very friendly, low-stakes way. Two informal data collection activities are also planned. The first orients you to the systematic collection of data to better understand some personal topic of interest to you. The second gives you a little taste of what it's like to interview someone. A course project is also required, though participants can choose among several options.
The philosophical foundation of the course is that there is not one method for research, but rather techniques that work better for a particular situation. As such, we will discuss the research process in the larger context of problem-solving. The overall goal is not to become an expert researcher, but to become more of an expert in the research process and a critical consumer of educational research. A knowledge of the research process, working in concert with tools and skills determined necessary for a given framework in a particular context, will enable you to accomplish this goal.
This is an online class. The overwhelming majority of the class content and activities are delivered or completed using asynchronous (not in real time) technologies. The advantage of this approach is that you can complete the course whenever and wherever you wish, according to a schedule determined by the instructor or a faster one determined by you. We will also meet occasionally online in a virtual classroom called Collaborate Ultra. The dates and times of these meetings can be found in the course schedule. The purpose of the virtual classroom meetings is not to deliver content, but to go over difficult concepts, enjoy some group interactions, and ask and answer questions. (If you are unable to attend any of the virtual class meetings, you will need to review the recording at your earliest convenience.)
Please note that this course syllabus is a general plan for the course; deviations announced to the class by the instructor may be necessary.
UGA Academic Honesty Policy
All academic work must meet the standards contained in "A Culture of Honesty." Students are responsible for informing themselves about those standards before performing any academic work.
This course follows the regulations outlined in the Americans with Disabilities Act. Call UGA Disability Services at (706) 542-8719 (voice) or (706) 542-8778 (TDD only) for information about architectural access and to arrange for sign language interpreters, assistive listening devices, large print, audio, or Braille. Students requiring special accommodation should contact the instructor as soon as possible.
Other UGA Policy Statements
UGA students working in schools are considered "mandatory reporters" under state law. This means that you are required by law to report suspected child abuse (for instance, if a student tells you about abuse or you suspect it based on a student's physical appearance or behavior). If you suspect child abuse, you must report this immediately to the school principal/site director, your UGA instructor, the UGA Police at 706-542-2200, and the Department of Children and Family Services at 1-855-GACHILD.
Leedy, P. D., & Ormrod, J. E. (2015). Practical research: Planning and design (11th ed.). Upper Saddle River, NJ: Pearson Education.
Note: The 10th edition is also acceptable. The 12th edition of this book has just been published, so feel free to purchase that edition instead. However, it is likely to be much more expensive. Used copies of the 11th and 10th editions will likely be readily available from online book sellers, such as Amazon.
Additional readings may also be assigned during the semester to supplement the text.
Participants in the course will become strategic consumers and proficient users of educational research. Participants will read and critique a variety of educational research studies that use various methods (quantitative, qualitative, mixed methods), with the objective of being able to analyze and apply research to professional contexts.
At the end of the course, each participant will be able to:
Things You Need to Already Know or Have
Other Things You'll Need
This is an all-online course with course content delivered primarily through asynchronous means. All of the course requirements (e.g. readings, software learning, project work, journal writing) will be done asynchronously using UGA's eLC learning management system. You access eLC at this web address:
After logging in, participants then navigate to the course home page. Participants need to log into eLC frequently throughout each week to access all course materials and to check on updates. The instructor will also post news and notes using the eLC News feature, found on the course's home page.
We will also meet regularly in a synchronous, "virtual" live classroom called Collaborate Ultra. The purpose of these synchronous class meetings will be to answer questions, resolve problems or confusion, to enhance understanding of key concepts, and to simply get together socially to remind ourselves that we are a group of "real" people working together to pursue a lifelong goal. The plan will be to meet online in the Collaborate Ultra virtual classroom for 60-90 minutes on Wednesdays at 6 pm.
However, these dates are tentative. Dates may change and more live classes may be added as needs arise. That said, "life happens." So, if you are unable to attend a live class despite your good faith efforts to do so, please know that each live class session will be recorded. If you are unable to attend a live class, you are responsible to watch the recording as soon as possible. (But, please know it is not the same as "being there.")
Communication among and between participants and the instructor is vital in any course, but particularly in an online course. The instructor will communicate primarily through two communications systems. The first is the UGA email system using each participant's UGA email address. Therefore, participants are required to check their UGA email account frequently. In addition, participants need to check their junk folder frequently as it is not uncommon for official email to be inadvertently tagged as junk. The only way to detect if this is happening is by manually checking one's UGA junk folder. (And, don't confuse the UGA email system with the email system contained within the eLC learning management system - these are two very separate systems. If you are confused about this, contact your instructor immediately.) The second is the "News" feature within our eLC course. Starting about day 3 of the course, all important course announcements, notes, and reminders will be distributed solely through the eLC "News" feature. So, you will either have to check the News feature at least once a day, or have all News items sent to you via email or through a text messaging service. Fortunately, this is very easy to do: After you log into eLC, click on the "notifications" option of your eLC account to set these preferences (look for your name in the upper right-hand corner of the eLC screen to access these options).
Statistics in Education for Mere Mortals (SEMM) Excel Exercises: Purpose: To learn some fundamentals of using statistics in education.
A series of friendly, low-states activities are planned to give you an introduction to some of the fundamentals of statistics. I use the term "mere mortals" is because these activities are designed for people who aren't, and don't want to be, statisticians or mathematicians. Instead, they are designed for ordinary people with ordinary math skills who want to learn the fundamentals of statistics well enough to apply them in their work. As the title also suggests, we won't take ourselves too seriously as we complete them.
You will build a series of Excel files that compute each of the following statistics:
These Excel files build upon each other. You will build these by watching and following a series of video demonstrations of me showing how to build the file in a step-by-step manner.
When you are done building your spreadsheet, your answers should, of course, match those shown in the video. Then, Lloyd will send everyone a new set of data to enter – all you will need to do is copy and paste the new data into the spreadsheet you just built. You’ll then be asked to submit some of the resulting statistics as an evaluation – just by copying and pasting from your spreadsheet into a web form.
The goal of these exercises is basically to do the computations by hand, but also at the same time taking full advantage of Excel. You will set up and define the calculations needed, but then let Excel do all of the tedious, hard work of actually performing the calculations. As a typical example, you will often need to enter a long column of test scores that must be added up, but you will actually have Excel do the calculation of summing.
We will deliberately avoid using any of Excel’s built-in statistical functions – which are many – until we have finished doing our own calculations. Then, we’ll use Excel’s stat functions to check our work. Remember, the goal of the course is to understand where all the calculated numbers and statistics come from. Getting the answer is not the point, but knowing how to get to the answer is.
These activities will be graded in the following manner. To get credit for an attempt on any Excel exercise, you must answer all of the questions correctly for that exercise. You can redo the exercise as often as you wish, with points awarded as follows:
After four attempts, your instructor will work with you to finish the exercise. No one will receive less than 75% credit for an exercise as long as they complete the exercise in whatever manner the instructor deems appropriate.
Note: You will have the option to continue your learning of statistics using this approach for the final course project, as explained below, to compute the following statistics:
Informal Activity - SDC - Systematic Data Collection: Purpose: To experience first-hand what it feels like to do collect data systematically on a topic of the participant's choice; To reflect on the role and value of research data to inform one's thinking or decision-making in practice.
As the name suggests, this is meant as an informal, fun assignment to gain some experience of what it feels like to actually do research on a topic of personal interest to you. Your task is to identify a subject or topic on which you have genuine curiosity and then collect data systematically on it. All you have to submit to get credit for this assignment is a brief written report (about 225 words) in which you describe the following:
(If you choose to collect data on yourself, a family member, or any other person, be careful not to disclose any personal information in your reflection given that this reflection will be posted on the Internet.)
We will brainstorm possible topics during class, but here are just a few to get you thinking:
Informal Activity - Conducting an Interview: Purpose: To gain experience with an essential qualitative data collection technique. You are free to interview any (willing) person, including a family member, on any appropriate topic. The interview can be short - I recommend aiming for 15 minutes. First, you need to prepare an interview protocol consisting of the basic questions you will ask. It is acceptable (and usual) that follow-up questions will be asked during the interview depending on answers given during the interview. However, it is important all of the questions on the protocol are asked. Audio record the interview. You do not have to transcribe the interview (though you would do so if this wasn't just an informal assignment), but you need to listen to it at least two times. Then, write a reflection (about 300 words) that covers the following points:
Research Design Activities: Purpose: To give each participant the opportunity to individually learn and implement the particular research design skill being introduced and discussed at that time; each RDA is focused on helping everyone understand equally the particular skill and process highlighted in the RDA.
RDAs are designed to enhance, extend, and support course content. These are completed individually and are meant to give you some initial practice in each of the major themes of the course. These activities are meant to be "low stress," so if you find yourself sweating over them, you are missing the point. The purpose of each RDA is to give you some quick, initial practice applying the respective design skill. If you do not complete a specific RDA appropriately, you will have one opportunity to resubmit based on the instructor feedback.
Though you are under no obligation to do so, it is perfectly acceptable to refine any or all of the RDAs to make them part of the research proposal project (option 1) if you choose that for your course project due at the end of the course. It is therefore wise to identify your project topic early on in order for you to use the RDAs as the first step in your project.
Two RDAs are planned based on the following topics:
Participants have their choice of three options for the course project: 1) Write a brief project proposal and record an accompanying presentation; 2) Focus on learning fundamental statistics in educational research; and 3) Focus on writing a review of literature on educational topic. Although Option 1 is the traditional project for this course, the other two options may be suitable for individuals with different learning goals. For example, individuals who intend to pursue a Ph.D. may find option 2 more relevant in order to extend their introductory knowledge of fundamental statistics. Finally, participants who have a strong interest in a particular research topic may find option 3 most useful and focus attention on reviewing the literature on that topic.
You need to make your decision by Friday, June 22. However, you can change your mind at any point, keeping in mind that all project options must be completed by July 13.
(Note: If you have already prepared a research proposal for another course on a topic that you wish to pursue, I am open to a fourth option where you actually conduct a small-scale pilot study following the procedures of that proposal. This is a good option for research-masters students who are about to start a Ph.D. program. If interested, contact me for us to discuss this option in more detail.)
Below are descriptions of each of these three options:
The task is to write a project proposal in which you apply your research skills. We will use a real-world analogy for the preparation of this project, that of persuading some administrative, governing, or funding group to take action in the form of approving (and perhaps funding) your project idea. Using that analogy, imagine that a short, allotted time has been given to you to address this group. Your project will consist of two elements:
Both elements need to describe or address the following: 1) Problem area or topic; 2) Review of the related literature; 3) Design of an approach you are taking to study, solve, or improve an educational problem.
The approach you are taking to solve or improve the problem can take one of two forms: An original research study proposal to collect new data on the topic; or a development project proposal to build or produce something on the topic with a design based on existing research.
Your presentation must be pre-recorded and made available to all participants in the course no later than July 9. (Tools and strategies for accomplishing this will be discussed in class.) The 2-page executive summary is due July 13.
This option constitutes a deep dive into the use of statistics in education and is a continuation of the SEMM activities you've already completed. This is a good option for anyone wanting to leave this course with some specific statistical skills useful in both evaluation and research. There are two requirements for this option. The first is to continue completing a series of activities that follow the same approach as the first three required SEMM activities to learn how to compute and use the following statistics:
Second, you need to find and summarize four examples of research articles that use any of these four statistical tests, or combination thereof, to analyze and interpret research data. Each entry consists of the following sections:
The most typical examples will be journal articles, but any credible scholarly source can be used. Other examples include reports by universities, institutes, government agencies, and professional organizations. You are cautioned against using examples from mass media but if you do so, be sure that the example followed rigorous review or fact-checking procedures.
With this option, you focus on reviewing the research literature on a topic of your choice. This option is a good choice for anyone who has already identified a strong interest in a particular topic and wishes to learn more about what research has already been done on it. This option is particularly well suited to anyone who is intending to apply to a Ph.D. program.
Your review of literature needs to be between 2000-2500 words (approximately 10 pages, double-spaced, not including references). It should follow this basic outline:
The paper needs to follow APA formatting guidelines.
Peer Critique of Research Design Project Presentations: Purpose: To watch and provide a brief critique of a subset of the presentations made by your classmates.
One of the time-tested benefits of this course is that participants tend to address very important and timely research topics in their final projects, resulting in an opportunity for the rest of us to learn a great deal. The requirements that all presentations be short and pre-recorded allow us to use our time flexibly and strategically to enjoy this benefit. Rather than be required to listen to every project presentation, each participant will instead be required to watch a subset that best matches their interests.
All participants must choose to watch at least 5 presentations of other classmates' research design projects and write and submit a brief critique of each (approximately 100 words for each critique) addressing the following points: 1) 1-sentence summary of the topic; 2) importance of the topic; 3) strengths and weaknesses of the project; and 4) persuasiveness of the presentation.
Details on how and where to submit these critiques will be announced and explained in class. All peer critiques are due no later than July 13, 2018.
Policy on Late Assignments
A goal of this course is to reduce anxiety and increase enjoyment in the learning of educational research methods. Consequently, there are no penalties for assignments turned in a little late. After all, life has a habit of intervening once in awhile. Here are the procedures you need to follow:
It should go without saying that you should take advantage of this late assignment policy sparingly. It meant for the occasional time when something in your personal life interferes with your completion of the course requirements.
Important note: This is a summer course with a condensed calendar. The rule of thumb is that two class days in the summer is equivalent to one week in a regular semester. Please plan your work and study plan accordingly, and expect to work every weekend.
Due dates below may be adjusted as needed. Any changes will be announced in email, then noted below.
Closing comments and cautions
This course, due to its strong application orientation, is highly activity-based. In completing the research design project, it is important that everyone identify their problem area early and pace their work appropriately. The research design project consists of many phases which build upon each other. Individuals who wait until late in the course to begin may experience "log jams" and may find it difficult to complete the project with high quality, thus risking a lower grade and, more importantly, less professional gain. Often, participants enter a course thinking that they can always take an "incomplete" if they decide they will not complete the course requirements on time. Keep in mind that I follow the university guidelines regarding an incomplete grade which reserve this for unforeseen circumstances or emergencies, not merely a failure to complete the work on time. I am confident that those who heed this warning will be able to complete the course with sufficient time to produce a high quality product while gaining personal and professional satisfaction.
This syllabus and the design of the course is adapted from courses taught by Dr. Janette Hill and Dr. Mary Ann Fitzgerald. I thank them very much for graciously offering their materials and expertise in the development of this course. I also acknowledge the efforts of Dr. Thomas Reeves -- his teaching of this course greatly informed the design and structure used by Dr. Hill and Dr. Fitzgerald. Finally, thanks also go to Dr. Michael Orey for sharing his ideas and experiences related to teaching this course.