ATC21S Project Phase 2.4 Field Test Revision

Appendix
A


Collaborative
Problem
Solving
Administration
Manual

Collaborative
Problem
Solving
Practice
Task

Collaborative
Problem
Solving
Peer
Survey

Collaborative
Problem
Solving
Self­Survey

Technical
Specifications


ATC21S
Collaborative
Problem
Solving
Assessment
Tasks

Administration
Guide
for
Teachers


This
document
is
the
administration
guide
for
Collaborative
Problem
Solving
Assessment
Tasks
for
use
by

teachers
in
Trials
in
2011.

Please
read
through
this
manual
prior
to
the
Trial
session
to
ensure
that
the

tasks
can
be
administered
successfully
and
consistently
for
all
students.


Sections:

1.

Introduction

1.1. About
Collaborative
Problem
Solving

1.2. About
the
tasks


2.

Planning


3.

Student
details
and
registering
for
CPS
tasks


4.

Administration
of
tasks


5.

Troubleshooting
guide


6.

Contact
Information


Section
1:

 
Introduction

The
ATC21S
project
is
developing
new
forms
of
assessment
and
teaching
approaches
to
meet
the
demands

of
the
21st
century.
One
of
the
areas
of
interest
is
the
assessment
and
teaching
of
collaborative
problem

solving.
The
goal
is
to
deliver
assessments
that
can
alert
teachers
to
appropriate
learning
interventions
and

give
instant
feedback
to
students
in
order
to
improve
their
collaborative
problem
solving
skills.


The
tasks
are
still
in
development
and
currently
do
not
provide
instant
feedback.
The
purpose
of
the
Trials

will
 be
 to
 finalise
 scoring
 rubrics,
 so
 that
 the
 Project
 can
 establish
 empirically
 based
 scales
 that
 have
 the

capacity
to
indicate
students’
location
on
the
developmental
progressions
associated
with
each
of
the
skill

sets.
 Student
 reports
 will
 be
 available
 from
 the
 assessments
 after
 the
 Trials.
 This
 information
 will
 allow

teachers
to
assess
their
students’
collaborative
problem
solving
capabilities.
The
Project
will
then
work
with

teachers
to
develop
teaching
interventions
for
students
operating
at
different
skill
levels.



Please
 note
 that
 access
 to
 the
 tasks
 is
 limited
 to
 participating
 countries
 under
 the
 management
 of
 the

National
 Project
 Managers.
 Login
 and
 password
 access
 should
 not
 be
 made
 public
 nor
 beyond
 those

involved
 in
 the
 formal
 research
 aspect
 of
 the
 ATC21S
 project.
 Any
 schools,
 teachers,
 and
 students

participating
are
to
be
given
access
only
for
the
purpose
of
data
collection.



2


1.1
About
Collaborative
Problem
Solving


Collaborative
problem
solving
(CPS)
is
conceptualised
as
being
composed
of
social
skills
and
cognitive
skills,

consisting
of
broad
strands
including
participation,
perspective
taking,
negotiating,
learning
and
knowledge

building.
 This
 broad
 structure
 has
 several
 elements
 which
 underpin
 the
 development
 of
 the
 assessment

tasks.


1.2
About
the
tasks

The
practice
task

The
 practice
 task
 is
 Light
 Box.
 This
 will
 help
 teachers
 learn
 the
 key
 features
 of
 the
 collaborative
 problem

solving
 assessment
 tasks
 and
 should
 be
 used
 with
 students
 before
 other
 tasks
 to
 help
 them
 understand

what
they
are
expected
to
do.
Please
refer
to
the
“Practice
Task
Guide
for
Teachers”
for
more
information

about
the
Light
Box.



The
assessment
tasks

Currently
there
are
eleven
assessment
tasks;
four
categorised
under
“Puzzles
and
Experiments”
and
seven

categorised
under
“Mathematical
and
Scientific”.



These
tasks
are
designed
to
elicit
collaborative
problem
solving
behaviours.

Students
work
in
pairs
on
the

tasks.
 
 In
 all
 tasks
 communication
 between
 partners
 is
 through
 on
 screen
 chat
 messaging.
 
 The
 tasks
 are

different
from
any
other
tasks
your
students
are
likely
to
have
used.

To
be
successful
students
must
work

together.
 
 The
 tasks
 require
 collaboration.
 
 The
 tasks
 are
 not
 timed
 and
 students
 are
 not
 rewarded
 for

finishing
quickly.

It
is
the
quality
of
the
student
interactions
that
is
important.




Early
 work
 has
 shown
 that
 students
 find
 the
 concept
 of
 assessing
 collaboration
 difficult
 to
 understand
 as

most
 assessments
 they
 have
 previously
 used
 have
 required
 them
 to
 work
 on
 their
 own.
 
 Many
 students

perceive
 it
 as
 ‘cheating’
 to
 ask
 their
 partner
 for
 help,
 or
 to
 discuss
 possible
 solutions
 with
 their
 partner.


With
these
tasks
it
is
precisely
students’
ability
to
work
together
to
solve
a
problem
that
is
being
assessed.




Success
in
these
tasks
is
achieved
through
active
participation,
working
out
what
the
problem
requires
and

identifying
 the
 information
 each
 partner
 holds,
 organising
 to
 work
 together,
 reading
 others’
 views
 and

suggestions
and
discussing
possible
approaches
through
text
chat,
as
well
as
of
course
finding
a
solution
to

the
problem
and
learning
from
the
experience.




Students
 can
take
 notes
 for
 themselves
 if
they
want
to,
on
paper.
However
all
communication
with
 their

partner
must
be
through
the
text
boxes
in
each
task.

It
is
important
that
students
are
not
close
enough
to

talk
to
each
other
as
this
verbal
communication
will
not
be
captured
for
analysis.


Student
survey

In
this
project
background
information
about
the
students
undertaking
the
assessment
tasks
will
be

collected.
These
details
will
include
student
age,
country,
first
language,
and
gender.
Students
will
also
be

asked
to
evaluate
their
performance
and
their
partner’s
performance.
This
information
will
be
collected

using
online
surveys
presented
to
each
student
upon
completion
of
the
tasks.


3


Section
2:

Planning

•
•
•
•
•

Please
read
through
this
manual
to
ensure
that
the
tasks
can
be
administered
successfully
and

consistently
for
all
students

Print
this
manual
if
you
are
reading
an
electronic
copy
as
you
will
need
a
paper
copy
during
the

administration
of
the
test

Check
if
technology
requirements
are
met
on
your
student
computers
(refer
to
document
“Technical

Specifications
and
Setup
Guide”)

Receive
your
student
logins
and
teacher
login
(contact
your
National
Project
Manager
[NPM]
if
you

have
not
received
these
–
contact
details
page
8)

Using
the
teacher
login
provided
to
you
by
your
NPM,
familiarise
yourself
with
the
login
area
and
the

practice
task,
as
students
may
request
assistance
to
help
them
initially
with
the
online
environment.




During
the
testing

•
•
•
•
•
•

Ensure
all
students
have
a
computer
and
access
to
the
internet.

Ensure
pencil
and
paper
is
available
for
students
to
take
notes

Ensure
all
students
have
comfortable
and
adequate
workspaces

Lighting
and
screen
brightness
should
enable
all
students
to
read
the
computer
screen
in
comfort.

There
should
not
be
shadows
or
glare
on
the
computer
screen
or
writing
surface

The
testing
room
should
comfortably
accommodate
the
number
of
testing
stations
placed
in
it

Testing
room
must
be
quiet
throughout
all
test
administrations.
When
testing
is
in
progress,
other

activities
that
would
disrupt
the
testing
environment
should
not
be
conducted


Section
3:


Student
Details
and
Registering
for
CPS
tasks

Before
beginning,
each
student
will
need
to
be
assigned
a
unique
login.
Please
refer
to
the
accompanying

login
spreadsheet.
There
are
two
pages:

“teacher
copy”
and
“student
copy”
(see
Figure
2).


1) In
the
teacher
copy,
assign
logins
to
students
by
writing
student
names
in
the
“student
names”

column.


2) Cut
the
student
copy
sheet
into
strips
containing
individual
logins
and
distribute
to
students
based

on
their
assigned
login
recorded
in
the
teacher
copy.


3) Do
not
intentionally
sit
students
with
the
same
team
codes
together.


Section
4:


Administration
of
Tasks

Currently
 there
 are
 eleven
 tasks;
 one
 Practice
 Task,
 four
 categorised
 under
 “Puzzles
 and
 Experiments”,

seven
categorised
under
“Mathematical
and
Scientific”.

Practice
task

Puzzles
and
Experiments

Mathematical
and
Scientific

Light
Box


Olive
Oil


Balance

Hexagons


Hot
Chocolate


Small
Pyramids


Sunflower


Laughing
Clown


Game
of
20


Plant
Growth


Shared
Garden


Warehouse



Each
task
is
part
of
a
bundle
with
other
tasks.
Students
complete
specific
bundles
of
tasks,
not
all
the
tasks.

The
 tasks
 listed
 against
 a
 login
 in
 the
 login
 spreadsheet
 define
 the
 bundle
 of
 tasks
 relevant
 to
 a
 student.


4


There
 is
 no
 expectation
 that
 all
 students
 will
 complete
 all
 their
 assigned
 tasks.
 Students
 have
 been

oversupplied
with
tasks
to
cater
for
those
who
complete
tasks
faster
than
others.



During
administration:
11‐step
guide
for
administration

This
guide
assumes:

•

•
•
•

Step
1

Step
2


Up
to
TWO
class
periods
will
be
available
per
bundle
of
tasks
(it
is
assumed
that
a
class
period
is
50‐
60
mins).
Periods
can
run
consecutively
or
with
a
break
in
the
middle
(e.g.
morning
tea
or
lunch

break).

Paired
students
will
undertake
tasks
simultaneously.

Please
ensure
those
in
a
pair
are
seated
far

enough
away
from
each
other
to
obstruct
oral
(voiced)
communication.

Pencil
and
paper
may
be
made
available
to
students.


To
ensure
session
runs
smoothly
and
efficiently,
where
possible,
have
all
computers
switched
on

and
ready
to
go
with
Internet
before
students
are
seated
for
the
session.


Write
the
website
http://www.arc‐alp.com/c21
on
the
board.

Hand
out
paper
student
login
strips.
Ensure
students
enter
their
login
and

password
as
it
appears
on
their
login
strips.

You
may
like
to
brief
students
about
the
ACT21S
Project
and
why
your
school
is

taking
part
or
examining
21st
century
skills
such
as
collaborative
problem
solving.


Read
out
the
instructions
to
students
given
below:


“You
are
about
to
take
part
in
a
trial
of
some
new
assessment
materials.
The
tasks

will
assess
how
well
you
can
solve
problems
working
with
a
partner.
It
is
more

important
that
you
work
together
than
it
si
that
you
solve
the
problems
on
your

own.


Sometimes
you
and
your
partner
will
see
different
objects
and
instructions
so
it
is

very
important
you
communicate
with
your
partner
using
the
on­screen
chat

messaging.


Never
share
your
thoughts
with
your
partner
by
talking
out
loud
to
them.


If
you
get
stuck,
ask
your
partner
if
they
have
information
that
may
help,
and
try

and
find
solutions
together.


You
will
now
have
10
minutes
to
do
a
practice
task.”


Ask
students
to
click
“Practice
Task.”


5


Please
note
any
irregularities
and
difficulties
you
experience
(such
as
technical
issues,
issues
with
this

Step
3

Step
4


Step
5


Ensure
students
select
Player
A
or
Player
B,
depending
on
what
is
on
their
login

strip.

Ask
students
to
begin
task,
advancing
through
until
the
pair
has
finished
all

sections.

If
students
need
task
assistance,
teachers
should
prompt
students
to

ask
their
partner.

If
students
need
further
assistance,
teachers
may
provide

group
guidance
to
the
whole
class
during
the
Practice
Task
ONLY.

When
10
minute
practice
session
is
up,
say
to
students:

“Click
browser
‘back’
button
until
you
exit
out
of
the
practice
task.

We
will
now
get
ready

to
do
the
assessment
tasks
listed
on
your
login
sheet.
Listen
carefully
to
my
instructions”



[Teacher:
point
to
relevant
sections
on
login
strip]
“You
will
start
with
the

first
task
listed
on
your
login
strip,
advancing
through
until
you
have

finished
all
sections.
After
that,
do
the
next
task
on
your
login
strip.
Make

sure
you
read
all
instructions.
Select
Player
A
or
Player
B,
depending
on

what
is
on
your
strip
.Do
not
rush.
Doing
the
tasks
well
with
your
partner
is

more
important
than
doing
lots
of
tasks.
You
can
take
notes
on
paper
if
you

wish.
Do
you
understand
what
I
want
you
to
do?
Are
there
any
questions?

You
will
have
two
periods
to
finish
all
the
activities.
You
may
start
now”.


If
students
need
task
assistance,
teachers
should
prompt
students
to
ask
their

Step
7
 partner.

If
students
require
I.T.
assistance,
please
help
students
accordingly.


If
there
is
a
break
between
period
1
and
2,
at
the
end
of
period
1
ask
students
to

Step
8
 log
out
by
clicking
‘Finish’
(where
available)
or
closing
browser
if
no
‘Finish’

button
is
available.


At
the
start
of
period
2
ask
students
to
log
back
in
and
continue
where
they
left

Step
9
 off.

When
there
is
15
minutes
left
in
period
2,
ask
students
to
click
“Finish”
or

Step
10
 browser
“Back”
button
to
return
to
main
menu.

Then
say:

“Now
please
click
on
‘Survey’.
You
will
be
given
two
short
on­
screen

questionnaires
about
the
task
and
how
you
got
on.

We
want
you
to
answer

these
questions
ON
YOUR
OWN.
Please
start
now.”


At
the
end
of
the
period:
“Thank
you
very
much
for
taking
part
in
this

Step
11
 project.

The
way
you
answer
the
tasks
will
be
used
to
make
the
tasks
better

for
teachers
and
students
all
over
the
world.
Please
hand
in
your
paper

notes
to
me”

Keep
a
look
out
for
students
who
have
been
waiting
a
long
time
to
connect


or
re­connect
with
their
partner.
When
both
students
get
disconnected,

they
may
both
get
message
“Waiting
for
your
partner
to
join”
or
similar,

and
may
end
up
waiting
indefinitely!
Ask
these
students
to
re­enter
the

original
URL
and
log
back
in
again.

Step
6


manual
or
tasks
etc.).
Please
forward
this
feedback
to
the
National
Project
Manager.

6


Section
5:
Troubleshooting
Guide

PROBLEM


POSSIBLE
SOLUTION


Both
students
from
a
pair
waiting
a
long

time
to
connect
to
their
partner,
with

message
“Waiting
for
partner
to
join”
or

similar.

Student
experiences
connection
issue
that

prevents
them
from
continuing
with
a

task
or
entering
a
task.



Both
students
need
to
log
out
then
back
in,
by

re‐entering
original
URL.



Student’s
chat
box
stops
updating
their

chat.

Text
is
too
big
for
the
screen
or
is

obscured
by
a
graphic
in
the
task.


Log
out
and
back
in,
by
re‐entering
original

URL.

Re‐size
the
screen.
To
make
the
screen

smaller
click
on
the
screen
then
hold
down

CTRL
key
while
pressing

–
key
as
many

times
as
required
.
To
make
the
screen
larger,

hold
down
CTRL
key
while
pressing
+
key
as

many
times
as
required.

Click
around
the
area
where
chat
box
should

be
and
it
will
appear.



Chat
box
is
not
visible.


Log
out
and
back
in,
by
re‐entering
original

URL.



Students
need
to
log
out
mid‐way
through
 Click
“Finish”
and
close
browser.
If
no

a
task.


“Finish”
button
is
available,
just
close

browser.


Students
need
to
navigate
to
main
menu

Click
“Finish”
where
available,
or
browser

mid‐way
through
a
task.


“Back”
button.


Students
encounter
“Page
cannot
be

found”
message
or
similar.


Both
students
from
a
pair
get
stuck
on
a

task
and
cannot
proceed
further
with
the

task.



Use
browser
“Back”
button
to
return
to
Main

Menu.

Ask
students
if
they
have
tried
seeking

information
and
help
from
their
partner.
If

so,
they
can
move
on
to
next
page.
If
on
last

page
of
task,
they
can
move
to
next
task.




Section
6:


Contact
Information

Your
main
contact
will
be
your
country
National
Project
Manager.
They
have
been
involved
in

the
project
for
many
months
and
will
able
to
answer
any
questions
that
are
not
in
this

document.



Contact
details
for
your
NPM
are:

Kathleen
Comfort

WestEd

730
Harrison
Street,


7


San
Francisco,
CA
94107‐1242

Ph:
707‐217‐7454

Email:
[email protected]

Administrative
Assistant:

Anna
Araneta

WestEd

300
Lakeside
Drive,
25th
Floor

Oakland,
CA

Ph:
510‐302‐4261

Email:
[email protected]

Associate
Professor
Esther
Care

International
Research
Coordinator
(IRC)

Assessment
&
Teaching
of
21st
Century
Skills
(ATC21S)

Assessment
Research
Centre

Melbourne
Graduate
School
of
Education
|
The
University
of
Melbourne
|
VIC
3010
Australia

T:
+61
3
8344
0975
|
F:
+61
3
8344
8739
|
M:
+61
431
657
341

E:
[email protected]
|
W:
www.education.unimelb.edu.au/arc


8


Figure
1.
An
example
of
part
of
a
login
spreadsheet
showing
the
“teacher
copy”
page.
Teachers

should
have
received
their
own
version
from
their
NPM.



Figure
2.
An
example
of
part
of
a
login
spreadsheet
showing
the
“student
copy”
page.
Teachers

should
have
received
their
own
version
from
their
NPM.


9


ATC21S
Collaborative
Problem
Solving
Assessment
Tasks

Practice
Task
Guide
for
Teachers

This
document
provides
teachers
with
a
guide
to
the
Collaborative
Problem
Solving
Practice
Task
(“Light

Box”)
for
use
in
Trials
in
2011.



About
the
practice
task

The
 practice
 task
 –
 “Light
 Box”,
 will
 provide
 an
 example
 of
 the
 key
 features
 of
 the
 collaborative
 problem

solving
 assessment
 tasks.
 Light
 Box
 should
 be
 attempted
 by
 teachers
 before
 the
 Trial
 session
 and
 by

students
before
they
attempt
any
of
the
assessment
tasks.
Should
students
need
task
assistance,
teachers

may
provide
guidance
at
group
or
individual
levels
during
the
practice
task
ONLY.
Thus,
teachers
will
need

to
 familiarise
 themselves
 with
 Light
 Box
 in
 order
 to
 answer
 any
 questions
 students
 may
 have.
 The

instructions
below
provide
guidance
to
teachers
for
accessing
and
completing
Light
Box.



Task
access

You
may
familiarise
yourself
with
the
task
by
having
yourself
and
a
colleague
login,
as
Teacher
Login
A
and

Teacher
Login
B,
respectively;
or
if
you
do
not
have
a
colleague
available,
you
can
login
using
two
browsers

or
computers,
as
explained
below.


1) In
one
browser
(eg.
Explorer),
go
to:
http://www.arc‐alp.com/c21/

2) Enter
Teacher
Login
A:


Student
ID:
[refer
to
login
given
to
you
by
NPM]

Team
Code:
[refer
to
login
given
to
you
by
NPM]


3) Fill
in
the
questionnaire
with
pretend
information
and
click
on
“Continue”

4) Click
on
“practice
task”

5) Click
on
Player
A


You
will
get
the
message:
“Waiting
for
your
partner
to
join…”


6) In
a
different
browser
(eg.
Firefox)
or
on
a
different
computer,
go
to:
http://www.arc‐alp.com/c21/


7) Enter
Teacher
Login
B:


Student
ID:
[refer
to
login
given
to
you
by
NPM]

Team
Code:
[refer
to
login
given
to
you
by
NPM]


8) Fill
in
the
questionnaire
with
pretend
information
and
click
on
“Continue”

9) Click
on
“practice
task”

10) Click
on
Player
B


Task
should
load
with
different
views
for
each
player
(browser)


10


Key
task
features

Light
Box
consists
of:

1) A
chat
box
for
students
to
communicate
with
their
partner,
located
on
the
bottom
right
hand

corner
of
the
screen.
Students
enter
messages
in
the
“input
message”
window
and
click
“send”
to

send
the
chat
messages
to
their
partner.
A
record
of
the
conversation
appears
in
the
rectangular

box
above
the
input
message
window.


2) A
task
environment
that
contains
a
different
view
for
Player
A
and
Player
B
and
different
task

resources
for
each
player.


3) Buttons
to
allow
players
to
move
on
to
next
page
or
exit
the
task
(players
cannot
go
back
to
a

previous
page
in
the
task).

Lightbox
has
two
“pages”
for
completion.


All
of
the
assessment
tasks
will
contain
these
three
basic
features,
although
they
may
differ

slightly
in
layout
and
appearance.



Figure
1.
Player
A’s
view
of
Page
1
when
chatting
is
in
progress.


11


Task
solution
–
Page
1

The
goal
of
the
task
is
for
both
players
to
place
their
individual
mirrors
in
the
correct
position

on
their
respective
grids
to
allow
the
laser
to
hit
the
target.
In
order
to
do
this,
Player
A
must

fire
the
laser
by
clicking
“Fire”
to
allow
players
to
see
the
path
of
the
laser
and
how
it
is
to
be

diverted
using
the
mirrors.
One
solution
is
for
Player
A
to
place
a
mirror
in
the
position

depicted
in
Figure
2
and
for
Player
B
to
place
their
mirror
in
the
position
depicted
in
Figure
3.

Players
must
communicate
with
each
other
to
identify
the
goal
of
the
task
as
well
as
to

coordinate
the
firing
of
the
laser
and
the
placement
of
the
mirrors.



Figure
2.
Player
A’s
view
of
Page
1
after
both
players
have
placed
their
mirrors
in
the
correct

position
and
laser
is
fired.


Figure
3.
Player
B’s
view
of
Page
1
after
both
players
have
placed
their
mirrors
in
the
correct

position
and
laser
is
fired.


12


Task
solution
–
Page
2

Again,
the
goal
is
to
divert
the
laser
into
the
target.
This
time,
the
path
of
the
laser
cannot
be

seen
and
only
one
mirror
is
required.

Players
receive
identical
tasks
which
they
are
to
solve

independently
of
the
other
player.
However,
if
their
task
solutions
do
not
match
they
will
be

told
“Your
answer
does
not
match
your
partner’s”.

They
may
choose
to
ignore
this
message
or

communicate
to
get
matching
answers.
Figure
4
depicts
the
correct
mirror
placement
by
both

players.
This
placement
leads
to
the
task
message
“Your
answer
matches
your
partner’s”.



Figure
4.
Player
A
and
Player
B’s
view
of
Page
2
after
both
players
have
placed
their
mirrors
in

the
correct
position.



13


Peer
Assessment

Question
When
did
your
partner
try
to
work
with
you?

Response
Options


·
Never

·
When
what
we
had
to
do
was
clear

·
When
what
we
had
to
do
was
not
clear

·
When
I
asked
for
information
or
ideas


Peer
Assessment


Question
Did
your
partner
reply
to
chats?

Response
Options


·
No

·
Yes


Peer
Assessment

Question
Did
your
partner
start
chats?

Response
Options


·
No

·
Yes


Peer
Assessment


Question
What
did
your
partner
do
when
you
were
doing
the
tasks?

Response
Options


·
Waited
for
me
to
explain
what
to
do

·
Asked
me
for
answers

·
Tried
to
do
the
tasks
alone

·
Worked
hard
with
me
to
solve
the
problem


Peer
Assessment

Question
How
did
your
partner
react
when
you
had
an
idea?

Response
Options


·
My
idea
was
ignored

·
My
idea
was
used
by
us
both

·
My
idea
was
used
along
with
my
partner’s
ideas


Peer
Assessment

Question
How
easy
was
it
for
you
to
understand
your
partner's
chat?

Response
Options


·
It
was
hard
to
understand

·
After
a
while
it
was
easy
to
understand

·
It
was
easy
to
understand
straight
away


Peer
Assessment


Question
What
did
your
partner
do
when
you
didn't
agree?

Response
Options


·
Ignored
me

·
Argued
with
me


14


·
We
eventually
agreed


Peer
Assessment


Question
How
good
is
your
partner
at
these
tasks?

Response
Options


·
Not
very
good

·
Good

·
Good
at
some
parts,
but
not
good
at
others


Peer
Assessment


Question
What
is
the
best
way
to
describe
the
way
your
partner
used
chat?

My
partner’s
chat
was
mostly…

Response
Options


·
not
there

·
telling
me
what
she/he
was
doing

·
asking
me
what
I
was
doing

·
checking
how
things
were
going


Peer
Assessment

Question
How
did
your
partner
plan
with
you?

My
partner…

Response
Options


·
started
before
we
were
ready

·
chatted
about
what
the
task
wanted
us
to
do

·
chatted
about
what
we
needed
to
do
together


Peer
Assessment


Question
Did
your
partner
chat
about
using
objects
to
solve
the
problem?

My
partner…

Response
Options


·
didn't
chat
with
me
about
it

·
chatted
about
what
to
use

·
chatted
about
how
to
use
things


Peer
Assessment

Question
How
did
your
partner
plan
the
approach
to
the
task?

My
partner…

Response
Options


·
tried
things
without
telling
me

·
told
me
what
she/he
was
going
to
do
next

·
told
me
all
the
things
she/he
was
going
to
do


Peer
Assessment


Question
How
did
your
partner
try
to
solve
the
task?

My
partner…

Response
Options
·
tried
different
things

·
tried
one
thing
at
a
time

·
checked
how
well
each
thing
worked


15


Peer
Assessment

Question
How
much
did
your
partner
help
to
make
the
problem
simple?

My
partner…

Response
Options


·
didn't
know
what
to
do

·
had
ideas
about
what
to
do

·
explained
exactly
what
we
needed
to
do


Peer
Assessment


Question
What
happened
when
you
and
your
partner
had
different
information
?

Response
Options


·
We
worked
on
our
own

·
I
asked
my
partner
for
information

·
My
partner
asked
me
for
information

·
We
shared
our
information


Peer
Assessment

Question
How
well
did
you
partner
explain
things
to
you?

My
partner…

Response
Options


·
was
hard
to
understand

·
made
some
sense

·
made
things
clear


Peer
Assessment

Question
What
happened
when
things
were
unclear?

My
partner…

Response
Options


·
got
frustrated

·
was
guessing
what
to
do

·
tried
different
things
to
see
if
they
would
work


Peer
Assessment


Question
What
happened
when
you
got
stuck?

My
partner…

Response
Options


·
tried
the
same
thing
over
and
over

·
tried
different
things

·
started
again
from
the
beginning
of
the
problem


Peer
Assessment


Question
What
did
your
partner
do
when
you
provided
information?

Response
Options


·
Used
one
piece
of
information
at
a
time

·
Used
many
pieces
of
information


16


·
Combined
all
the
information


Peer
Assessment

Question
What
did
your
partner
do
when
you
needed
more
information?

Response
Options


·
Waited
for
me
to
say
what
I
needed

·
Sent
me
one
sort
of
information

·
Tried
giving
me
different
sorts
of
information


Peer
Assessment

Question
What
happened
when
you
had
lots
of
confusing
information?

My
partner…

Response
Options


·
wrote
everything
down

·
listed
the
main
things

·
reorganised
the
information


17


Self
Assessment


Question
When
did
you
try
to
work
with
your
partner?

Response
Options


·
Never

·
When
what
we
had
to
do
was
clear

·
When
what
we
had
to
do
was
not
clear

·
When
he/she
asked
for
information
or
ideas


Self
Assessment

Question
Did
you
reply
to
chats?

Response
Options


·
Yes

·
No


Self
Assessment


Question
Did
you
start
chats?

Response
Options


·
Yes

·
No


Self
Assessment

Question
What
did
you
do
when
you
were
doing
the
tasks?

I…

Response
Options


·
waited
for
my
partner
to
tell
me
what
to
do

·
asked
my
partner
for
the
answers

·
had
to
do
the
tasks
on
my
own

·
worked
hard
with
my
partner
to
solve
the
problem


Self
Assessment


Question
How
did
you
react
when
your
partner
had
an
idea?

I…

Response
Options


·
ignored
his/her
ideas

·
used
his/her
ideas

·
used
my
ideas
with
his/her
ideas


Self
Assessment

Question
How
easy
was
it
for
your
partner
to
understand
your
chat?

Response
Options


·
My
partner
did
not
seem
to
understand
me

·
I
had
to
say
the
same
thing
in
different
ways

·
I
only
had
to
say
things
once
to
be
understood


18


Self
Assessment


Question
What
did
you
do
when
you
didn't
agree?

Response
Options


·
I
ignored
my
partner

·
I
argued
with
my
partner


·
We
eventually
agreed


Self
Assessment


Question
How
good
are
you
at
these
tasks?

I
am
…

Response
Options


·
not
very
good

·
good

·
good
at
some
parts,
but
not
good
at
others


Self
Assessment

Question
What
is
the
best
way
to
describe
the
way
you
used
chat?

I
mostly...

Response
Options


·
did
things
by
myself

·
told
my
partner
what
I
was
doing

·
asked
my
partner
what
she/he
was
doing


Self
Assessment

Question
How
did
you
plan
with
your
partner?

I…

Response
Options


·
did
not
read
the
instructions

·
talked
about
what
the
task
wanted
us
to
do

·
talked
about
what
we
needed
to
do


Self
Assessment


Question
Did
you
chat
about
using
objects
to
solve
the
problem?

I…

Response
Options


·
didn’t
chat
about
it

·
chatted
about
what
to
use

·
chatted
about
how
to
use
things


Self
Assessment


Question
How
did
you
plan
the
approach
to
the
task?

I…

Response
Options


·
clicked
on
different
things
to
see
how
they
work

·
thought
about
what
I
was
going
to
do
next

·
thought
about
all
the
steps
I
needed
to
follow


19


Self
Assessment


Question
How
did
you
try
to
solve
the
task?

I…

Response
Options


·
tried
different
things

·
tried
one
thing
at
a
time

·
checked
how
well
each
thing
worked


Self
Assessment

Question
How
much
did
you
help
to
make
the
problem
simple?

I…

Response
Options

·
didn't
know
what
to
do

·
had
ideas
about
what
to
do

·
knew
exactly
what
to
do


Self
Assessment


Question
What
happened
when
you
and
your
partner
had
different
information?

Response
Options


·
We
worked
on
our
own

·
My
partner
asked
me
for
information

·
I
asked
my
partner
for
information

·
We
shared
our
information


Self
Assessment


Question
How
well
did
you
explain
things
to
your
partner?

I…

Response
Options


·
found
it
very
difficult

·
found
it
hard
for
some
things

·
found
it
easy


Self
Assessment

Question
What
happened
when
things
were
unclear?

I…

Response
Options


·
got
frustrated

·
guessed
what
to
do

·
tried
different
things
to
see
if
they
would
work


Self
Assessment

Question
What
happened
when
you
got
stuck?

I…

Response
Options


·
tried
the
same
thing
over
and
over

·
tried
different
things


20


·
started
again
from
the
beginning
of
the
problem


Self
Assessment

Question
What
did
you
do
when
your
partner
provided
information?

I…

Response
Options


·
used
one
piece
of
information
at
a
time

·
used
many
pieces
of
information

·
combined
all
the
information


Self
Assessment


Question
What
did
you
do
when
your
partner
needed
more
information?

I…

Response
Options


·
waited
for
my
partner
to
tell
me
what
was
needed

·
sent
one
sort
of
information

·
tried
giving
different
sorts
of
information


Self
Assessment


Question
What
happened
when
you
had
lots
of
confusing
information?

I…

Response
Options


·
wrote
everything
down

·
listed
the
main
things

·
reorganised
the
information


21


ATC21S
Collaborative
Problem
Solving
Assessment
Tasks

Technical
Specifications
and
Setup
Guide


This
document
is
for
technical
staff
responsible
for
setting
up
the
assessment
tasks
on
school
computers

during
Trials
in
2011.
Please
read
and
follow
the
instructions
in
this
document
before
your
school
is
due
to

participate
in
the
Trials,
in
case
you
experience
any
unexpected
technical
issues.



Technical
specifications


Below
are
hardware,
software
and
IT
requirements
for
the
Collaborative
Problem
Solving
Assessment

Tasks:



Technical
specifications

•
•
•
•
•
•
•
•
•

computer
is
needed
for
each
student


devices
supported
‐
PC
or
Mac


browsers
‐
PC:
Microsoft
IE
7.0+
or
FireFox
3.0+;
Mac:
Safari
4.0+
or
FireFox
3.0+

browser
settings
‐
javascript
and
pop‐up
windows
must
be
enabled

other
capabilities
‐
java
1.6+
installed
and
enabled

plugin
‐
Adobe
Flash
10.3

clear
browser
caches
prior
to
test
administration

Internet
connectivity
‐
broadband
1.5Mbit/s
or
higher
recommended

screen
size/resolution
‐
1024x768
or
higher
recommended,
works
at
less


Access
to
external
website

The
assessment
tasks
require
access
to
an
external
website:


http://www.arc‐alp.com/c21


Please
ensure
that
proxy
settings
and
filters
do
not
block
this
site
and
its
components
during
the
Pilot.
You

will
need
to
follow
the
instructions
in
the
next
section

(“Test
for
site
access”)
to
verify
that
the
system
will

work
at
your
school.



22


Test
for
site
access

To
test
for
site
access,
please
follow
these
instructions:


11) In
one
browser
(eg.
Explorer),
go
to:
http://www.arc‐alp.com/c21/


12) Enter
Teacher
Login
A:


Student
ID:
[refer
to
login
given
to
you
by
NPM]

Team
Code:
[refer
to
login
given
to
you
by
NPM]


13) Fill
in
the
questionnaire
with
pretend
information
and
click
on
“Continue”


14) Click
on
“Mathematical
and
Scientific”


15) Click
on
“Balance”


16) Click
on
Player
A


You
will
get
the
message:
“Waiting
for
partner
to
join…”


17) In
a
different
browser
(eg.
Firefox)
or
on
a
different
computer,
go
to:
http://www.arc‐alp.com/c21/


18) Enter
Teacher
Login
B:


Student
ID:
[refer
to
login
given
to
you
by
NPM]

Team
Code:
[refer
to
login
given
to
you
by
NPM]


19) Fill
in
the
questionnaire
with
pretend
information
and
click
on
“Continue”


20) Click
on
“Mathematical
and
Scientific”


21) Click
on
“Balance”


22) Click
on
Player
B


Task
should
load
with
different
views
for
each
player
(browser)


23) Click
browser
“Back”
button
to
return
to
Main
Menu
in
both
browsers


24) Click
on
“Puzzles
and
Experiments”
in
both
browsers


25) Click
on
“Olive
Oil”
in
both
browsers


26) Click
on
Player
A
in
one
browser
and
Player
B
in
the
other
browser


Task
should
load
with
different
views
for
each
player
(browser)


If
there
are
connection
issues
during
any
of
the
above
steps,
please
contact
your

National
Project
Manager
(NPM)
as
soon
as
possible:


NPM:
enter
your
contact
details
here.



23