Evaluating Evidence

ISS 305:003 – Evaluating Evidence
Assignment #3
Due Dates and Grades
• Assignment #3 is due at 4pm on 12/1/22 (a Thursday) to the D2L paper folder.
• This assignment is worth 50 points (10% of your final grade).
• Papers turned in after 4pm on 12/1/22 will receive a 10 point deduction for each day they are late (e.g., a paper turned
in 2 days late will be marked down 20 points). I include Saturday and Sunday when I count the number of days the
paper is late.
Overview
The purposes/goals/objectives of this paper is to describe, interpret, report on, and assess the replication crisis.
Specifically, you will decide what field and research design is more susceptible to the replication crisis and the problems
associated with it, and perform an interview with a friend or family member where you apply the major concepts and
theories relevant to ISS 305 that you have learned so far. This assignment covers ISS Undergraduate Learning Goals
#1, #2, and #5.
Guidelines for the paper
To earn the highest possible score, be sure to do the following:
• Address #s 1-5.
• Write the paper in a numbered format, corresponding to each of the numbered questions for you to answer.
• Throughout the paper, use details to support your main argument and repeatedly link to material covered
throughout the course. There are multiple places links can, and should, be made.
• For full points, be thoughtful, organized, have good sentence and paragraph structure, and be free of punctuation
and spelling errors.
• You do not need a separate introduction or conclusion section of the paper. Just provide responses to the
numbered prompts.
• Be sure to work independently and present only your own work. Plagiarism (using someone else’s words or
ideas without giving them credit) will result in 0 points for the paper and course.
o Besides when directly quoting your friend or family member, do not take direct quotes and
insert them in as answers. You need to develop original work and content for this course. It is
expected that you gather information and display it in a manner that isn’t just copying and
pasting it into your paper. Cite the work (see FAQ #5), but do not copy-and-paste the original
content.
o Turnitin will be used. See the syllabus for more details.
Papers should be:
• 2-4 pages in length (maximum 5 pages)
• Double-spaced, 12 point Times New Roman font with 1-inch margins on all sides
• Do not include identifying information in your paper (e.g., name or student number).
• Upload your paper to the D2L paper folder.
Papers do not need to be:
• In APA format, MLA, Chicago style, or whatever style you have been taught. If you want to include citations and
a reference section, great! In fact, you are encouraged to do so! However, the formatting choice of the citations
and the reference section are up to you.
A FAQ is provided at the end of this document (after the rubric).
1. Do you think the problems discussed in the “Real vs. Illusory Effects” lecture series (especially in Part 7)
about the replication crisis, file drawer problem, multiple comparisons, and p-hacking are particularly likely in
a certain area/field of research? Why? Or, why not? What about a certain research design (survey, experiment,
etc.)? Why? Or, why not? Apply course material to help support your answer.
2. Have a friend or family member read this article – https://www.vox.com/future-perfect/21504366/sciencereplication-crisis-peer-review-statistics. Then interview them. You can ask any questions you like, but get a sense
of if they sound like a Candide, skeptic, or Skeptic after reading the article. Then explain why you believe they fit
into one of the three categories. Apply course material to help support your answer.
3. Give your friend or family member one piece of advice from the course that you think is the best advice for
approaching new evidence. Specifically, what is something from the course that you would tell them to use when
evaluating evidence in the future and why? Apply course material to help support your answer.
4. What did your friend or family member think about the advice you gave them in #3? Did the advice change their
mind for how they will approach new evidence in the future? If yes, how so and why? If no, why not?
5. Imagine that you and a group of people are presented with some new evidence trying to convince you of
something. (Could be anything – taking a new medication, buying a certain kind of car, which dog breed is the
best, etc.) Based on your interview and discussion with your friend or family member, how will you approach
talking about evaluating new evidence with this group of people? Specifically, what is your strategy for how to
discuss “becoming a better research consumer” with them? Apply course material to help support your
answer.
Paper Rubric
Criteria Failing Below Average Average Very Good /
Strong
Exceptional / Superior
#1
10 pts possible
0 – 3 points – No
Answer provided, or
answer provided has
nothing to do with the
questions asked (0
points); area/field and
design mentioned but
no supporting details
are provided (1
points); area/field and
design mentioned are
presented, but very
minimal supporting
details are provided (2
– 3 points)
4 – 5 points –
Area/field and
design are
mentioned, but the
supporting details
for why (or, why
not) are minimal.
6 – 7 points –
Area/field and
design are
presented and
supporting
details are
provided, but
lack in
development.
Specifically,
only surface
level details for
why (or, why
not) are
provided.
8 – 9 points –
Area/field and
design are
presented and
supporting details
are provided,
appropriate, and
are past surface
level details; but
still lacking in
full development
for why (or, why
not).
10 points – Area/field
and design are presented
and supporting details
for why (or, why not)
are fully developed and
appropriate.
#2
5 pts possible
0 – 1 points – No
answer provided, or
answer provided has
nothing to do with the
question.
2 point – Some
description of why
the friend or family
member fit into one
of the three
categories is
included, but no
clear supporting
details.
3 points –
Description of
why the friend
or family
member fit into
one of the three
categories is
included, but
only surface
level details are
provided.
4 points –
Description of
why the friend or
family member
fit into one of the
three categories
is included and
supporting details
are provided,
appropriate, and
are past surface
level details; but
still lacking in
full development.
5 points – Clear
description of why the
friend or family member
fit into one of the three
categories is included.
The supporting details
are fully developed and
appropriate.
#3
5 pts possible
0 – 1 points – No
answer provided, or
answer provided has
nothing to do with the
question.
2 point – Some
description of the
“best advice” is
included, but no
clear supporting
details.
3 points –
Description of
the “best
advice” is
included, but
only surface
level details are
provided.
4 points –
Description of
the “best advice”
is included and
supporting details
are provided,
appropriate, and
are past surface
level details; but
still lacking in
full development.
5 points – Clear
description of the “best
advice” is included. The
supporting details are
fully developed and
appropriate.
#4
5 pts possible
0 – 1 points – No
answer provided, or
answer provided has
nothing to do with the
question.
2 points – Some
description of what
the friend or family
member thought
about the advice is
included, but no
clear supporting
details.
3 points –
Description of
what the friend
or family
member thought
about the advice
is included, but
only surface
level details are
provided.
4 points –
Description of
what the friend or
family member
thought about the
advice is
included and
supporting details
are provided,
appropriate, and
are past surface
level details; but
still lacking in
full development.
5 points – Clear
description of what the
friend or family member
thought about the advice
is included. The
supporting details are
fully developed and
appropriate.
#5
5 pts possible
0 – 1 points – No
answer provided, or
answer provided has
nothing to do with the
question.
2 points – Some
description of how
to approach
“becoming a better
research consumer”
with other people is
included, but no
clear supporting
details.
3 points –
Description of
how to approach
“becoming a
better research
consumer” with
other people is
included, but
only surface
level details are
provided.
4 points –
Description of
how to approach
“becoming a
better research
consumer” with
other people is
included and
supporting details
are provided,
appropriate, and
are past surface
level details; but
still lacking in
full development.
5 points – Clear
description of how to
approach “becoming a
better research
consumer” with other
people is included. The
supporting details are
fully developed and
appropriate.
Incorporation
of information
from lecture
and the
readings
15 pts possible
0 – 3 points – No
attempt to incorporate
information from the
course material (0
points).
One to two minimal
attempts are made
(e.g., just providing a
concept’s name) to
provide linking
material (1 point).
Three to four minimal
attempts are made (2 –
3 points).
4 – 9 points –
Attempts are made
to incorporate
linking materials,
but are:
1) Not used
correctly and
lacking in detail and
depth (4 – 5 points).
2) Mostly used
correctly, but
lacking in detail and
depth (6 – 7 points).
3) Used correctly,
but lacking in detail
and depth (8 – 9
points).
10 – 11 points –
Links are made
with the course
material and are
used correctly,
but supporting
details did not
sufficiently
make the
appropriate
linked point.
12 – 13 points –
Links are made
with the course
material and are
used correctly,
but some
supporting details
are not always
relevant and do
not always hit the
mark of fully
developed
incorporation.
14 – 15 points – Links
are clearly made with
the course material, used
correctly, relevant, and:
Supporting details are
provided (14 points).
Supporting details are
fully developed and
appropriate (15 points).
Formatting /
organization,
grammar /
spelling,
overall quality
5 pts possible
0 – 1 points –
Formatting guidelines
not followed; and
many problems with
sentence structure,
grammatical errors, or
spelling errors.
2 points – Either:
Formatting
guidelines not
followed. Or, many
problems with
sentence structure,
grammatical errors,
or spelling errors.
3 points –
Formatting
guidelines
somewhat
followed; and
there are a few
problems with
sentence
structure,
grammatical
errors, or
spelling errors.
4 points –
Formatting
guidelines
followed, but
there are a few
problems with
sentence
structure,
grammatical
errors, or spelling
errors.
5 points – Formatting
guidelines followed.
There may be one or two
mechanical errors that
are not serious.
Frequently Asked Questions (FAQ)
1) What does “numbered format” mean?
a. Numbered format means to put in the numbers and respond to those prompts. For example, this FAQ is in
a numbered format.
2) Does the paper actually have to be 2-4 pages in length?
a. Not necessarily. Not including a page length freaks students out. I’m making a guess on the page length
based on past assignments in this course.
3) The paper is worth 50 points. How are the points broken down (i.e., what should I focus on)?
a. See the rubric that is provided and pay special attention to the parts that are worth the most. Incorporation
of course information is worth 15 points, or 30% of the assignment!
4) Can we use quotes?
a. Besides when directly quoting your friend or family member, do not take direct quotes and
insert them in as answers. In general, you should shy away from direct quotes in your writing. You
should be gathering information and organizing it in a way that can be easily digested by a reader.
Therefore, use your own words with citations (see FAQ #5). People tend to overuse quotes and it makes
documents hard to navigate, and get a good sense if you actually understand the course material. Work on
your writing by summarizing the work into your own words.
5) How do you want us to cite the readings or the lectures?
a. This course is not designed to teach you how to use APA formatting, MLA, Chicago style, etc. Therefore,
most of the formatting is up to you, but please note the “Papers should be:” section found on page 1 for
more information on what is required for formatting. You can easily say things like, “As pointed out in
lecture”, “As referenced in X reading”, “As Dr. Weaver so amazingly explained”, etc.
6) How many links / applications to the material is enough?
a. This is tough to answer. You could link to five things and go really in-depth and be good. Or, you could
just say five vocabulary type words from the course and do nothing else, which would not be good. You
need to show application of the material into the assignment and that you understand the material you are
using. For example – Was enough information provided to make the linked point? Was it correctly used?
7) What kind of links / applications to the material are you looking for?
a. A good place to start is to think about the major concepts or theories we have covered. Like – the
replication crises and the problems associated with it, fallacies, heuristics, etc. Then you could incorporate
examples or findings that were covered to drive home the point you are making.
b. Note that some links are not as impactful as others. For example, pointing out that heuristics are “rules of
thumb” is not an impactful link, but explaining how or why or when that might influence decisions,
perspectives, etc. is impactful if correctly used.