Does peer review work? Thoughts on conference session selection.

So here’s something interesting from my inbox!

Only 8% members of the Scientific Research Society agreed that “peer review works well as it is.” (Chubin and Hackett, 1990; p.192).

“A recent U.S. Supreme Court decision and an analysis of the peer review system substantiate complaints about this fundamental aspect of scientific research.” (Horrobin, 2001).

Horrobin concludes that peer review “is a non-validated charade whose processes generate results little better than does chance.” (Horrobin, 2001). This has been statistically proven and reported by an increasing number of journal editors. (full text and references)

I got this email from the International Symposium on Peer Reviewing: ISPR being organized in the context of The 3rd International Conference on Knowledge Generation, Communication and Management: KGCM 2009, which will be held on July 10-13, 2009, in Orlando, Florida, USA.

I assume that I got on their mailing list because of my post from almost a year ago, Conference 2.0 – changing how sessions are selected. In that post, I mentioned the KGCM conference and how these experts are investigating how people learn at conferences and are questioning how conference sessions are selected.

In the past year, I’ve read many blogs wondering about conference sessions and the selection process. Many use a form of “peer review”, meaning that people who are presumably peers of the paper author or conference session presenter rate the submission and determine whether or not it gets selected. Academic journals use peer review as a means to claim that their articles are superior to those found in lay magazines, books, or other sources. And many conferences use this same technique to select sessions, whether or not there are academic proceedings that result from the conference.

So this research not only calls into question the methodology of selecting conference sessions, it questions that whole academic tradition of journals representing the highest level of publishing and “truth” about current research in any field. They find that the principle of peer review is well-regarded, but that the methodology and implementation is flawed. Doesn’t that sound familiar…

If peer review is a flawed process, what does that say about conference session selection? You have to ask yourself, would a completely open selection process, where everything is out in the open be more in keeping with the principle of fairness and making sure best practices and research are shared to benefit all?

Would an open selection process for conference presentations be more or less prone to favoritism, self-promotion, populism, and cronyism than the current process? Is that just throwing the baby out with the bathwater?

If you are interested in joining a conversation about improving conferences, check out this social network created by the Association for the Advancement of Computers in Education (AACE) — Spaces of Interaction. They are having an online conversation Feb 18-20 with interactive live sessions.

And by the way, what does this say about school librarians who insist that students use only peer-reviewed journal articles for their research. Do they have a leg to stand on anymore?

Sylvia

The Parent-Teacher Talk Gains a New Participant – NYTimes.com

The Parent-Teacher Talk Gains a New Participant – NYTimes.com.

Today’s New York Times features an article about student participation in teacher-parent conferences. Good news: good facts and stories woven into a compelling article that supports student empowerment by including them in their own education. Bad news: it’s not new, and leaves out the mountains of research and practice about student conferences.

But, Google to the rescue! Here’s a great collection of resources from Education World (and hurray, recently updated) about how to plan and implement successful parent-student-teacher conferences.

Sylvia

Constructing Modern Math/Science Knowledge 2009

Ever question why technology seems to have gone missing in so many math and science classrooms? What happened to the “compute” in computing? Wondering what STEM really looks like?

Yes, technology, math, and science can be friends!

Constructing Modern Knowledge is organizing a one-of-a-kind educational event for January 22, 2009 at Philadelphia’s Science Leadership Academy. Constructing Modern Math/Science Knowledge is a minds-on institute for K-12 teachers, administrators and technology coordinators looking for practical and inspirational ways to use computers to enhance S.T.E.M. learning. Constructing Modern Math/Science Knowledge is a pre-conference event for Educon 2.1, an innovative conference and conversation about the future of education.

The presenters represent high-tech pioneers and seasoned veterans at the forefront of innovation in math, science and computing. Read more about them here.

Come to Constructing Modern Math/Science Knowledge and stay for Educon 2.1!

  • Early-bird registration (before December 15) – $100
  • Regular registration – $130

You may register for both Constructing Modern Math/Science Knowledge and Educon 2.1 with one click.

Sylvia

Back to New York and NYSCATE

Well, it seems like I just got home from the east coast, and I’m off again!

This time I’m headed for the New York State education technology conference NYSCATE in Rochester, NY November 23-25, 2008. I’m looking forward to seeing old friends and meeting new ones, most likely at Dinosaur BBQ.

If you are going to NYSCATE, be sure to check out these sessions:

NYSSTL –Technology Leadership for the 21st Century
Sunday, 1:45PM Stacy Ward
Learn how the HFM and WSWHE BOCES have created the New York State Student Technology Leaders (NYSSTL) Club in 30 middle schools. Students help their teachers learn to use technology and their classmates prove their tech literacy, creating a community of 21st century learning in our schools.

Where Teachers Learn, Where Teachers Teach
Monday, 10:45AM Sylvia Martinez
For many teachers, technology professional development happens outside the classroom and never crosses the doorstep into the classroom. This session will explore two models of professional development that cross that barrier: classroom embedded and student-led professional development.

Little Green Monsters: The XO and Its Implication For Education
Tuesday 10:30AM Brian C. Smith, Sylvia Martinez, Dr. Gary Stager
The XO low cost laptop was designed to revolutionize education in the developing world. The panel will discuss the lessons we can gain from this learning initiative and the implications for the future of education. We will also explore why such a simple idea has created such controversy.

By the way, I’m happy to have someone record, live blog, or ustream my sessions IF you can come and do it. It’s just too hard to do it AND present.

After that, it’s back to New York City for a family/friends Thanksgiving, and then some workshops in Brooklyn. More about that later!

Sylvia

Subscribe to the Generation YES Blog

A trip to Open Space!

Well, it seems like I just got back from T+L in Seattle and I’m off again.

First to Chester County in Pennsylvania where I’ll be meeting with several schools about our student technology programs. One of their districts, Coatesville, will be using TechYES Science for student tech literacy certification using science projects. I’m looking forward to meeting up with some PA friends as well.

Then a couple of stops in New Jersey and on to New York. I’m honored to be a featured speaker at the New York State Association of Independent Schools (NYSAIS) Educational Information Technology Conference, NEIT 2008.

“NEIT 2008 is an unconference planned for technologists and librarians. Our unconference relies on the open space technology for creating workshops, discussions, breakout groups, etc.”

This is the first time I’ve heard of open space technology. Wikipedia describes it as a method of self-organizing that can be applied to gatherings of any size to quickly tackle complex issues. It sounds a bit like FutureSearch, which I’m more familiar with, but definitely something worth knowing about!

The EdTechTalk podcast this week featured a conversation about the plans and goals for the NYSAIS event.

Plus, this conference is at the beautiful Mohonk Mountain House and you can’t beat that for open space!

Sylvia

Subscribe to the Generation YES Blog

Community of interest or community of practice?

I’ve been seeing a lot of talk around the edu-blogs and at conference sessions about online learning communities, or building a personal learning network as part of a educator’s professional development. Often, these are referred to as “Communities of Practice” – a term coming into common use only a few years ago. Many educators were introduced to the term in grad school through the work of Jean Lave and Etienne Wenger, who wrote Situated Learning: Legitimate Peripheral Participation.Learning in Doing: Social, Cognitive and Computational Perspectives (Amazon link) in 1985.

Their book explored how natural learning that takes place in apprenticeship situations, and profiled several different Communities of Practice (CoPs) from around the world. The “practice” part of CoP is the work they do, and the learning takes place in context, or situated, in the common work. These groups of people learn to do their work not by lectures, but by everyone working together, from experts to newcomers, and most importantly, by talking about their work together.

The concept of “Legitimate Peripheral Participation” is key to the idea of communities of practice. This is when newcomers learn needed skills by doing work that is on the periphery of the community, and as they prove their competence, being invited into more important roles. The other part of legitimate peripheral participation is how newcomers move into the community through talk. The shared stories of the community, particularly war stories told by experts, are part of the experience. Newcomers learn to participate by learning the traditions and vocabulary of the group, first by listening, then by trying out their new verbal skills within the group, and if their words ring true, by moving from the periphery and becoming more central to the shared expertise of the group.

It’s obvious that this sounds similar to what happens to educators as they break down the walls of their classroom and use new technologies to find and participate in new communities.

I think, however, that we confuse different kinds of communities. Specifically, I think that educators who come together in online communities, or even temporary real life groups, are more often than not, communities of interest, not communities of practice.

I’m not just being picky about semantics here – the problem is that calling any community a “Community of Practice” presumes that it will have similar benefits and impact.

In Second Life, for example, a teacher may meet educators from around the world who are doing similar things in their classroom, have similar hopes about the future of ed tech, and share similar frustrations. They may find the interaction refreshing, educational, and maybe even inspiring. These professional collegial interactions are too often missing from teacher’s lives. And Second Life is just an example. This could be Twitter, or a social network, an online group created for a graduate school class, or even people you meet face to face at a conference.

However, just having similar work doesn’t mean that this is a community of practice. They really aren’t doing the same work. Once this interaction is over, they have to go back to their real place of practice, their classroom and school. The benefits of shared vocabulary, shared experiences, shared stories are all gone. Now these teachers have to sit in meetings where no one is on the same page, frustrated that everyone isn’t seeing the light.

In fact, these outside communities of interest may serve to pull teachers away from their local communities of practice, distancing them from the colleagues whose mindshare would be vital to real local change. It’s an all too convenient place to vent about everyone who “doesn’t get it.”

It’s hard for me to imagine any kind of educational change that doesn’t draw on the participants at the ground level, meaning the people in the school. That’s why I advocate for student participation in education technology initiatives. Changing a community means involving the stakeholders, all the stakeholders, in the process. Building a healthy community of practice in the place of the actual practice is a first step to change.

 

K12online 2007 is still online!

The other day I mentioned the K12 Online Conference 2008, and my Games in Education presentation.

But what I forgot to mention is that previous year’s sessions are still available, and still really relevant to educational technology! On the K12online conference website just look down the right sidebar and you will see the links to the archives of all the sessions from the previous years of 2006 and 2007.

Last year I participated in the K12online conference with two sessions and contributed to a third (see K12 Online 2007 Conference – comfy slipper learning for the whole story).

This year I came to my senses and only submitted one idea for a session about games in education. For those of you who don’t know, in two of my many previous careers I was an executive producer for Davidson & Associates (now Knowledge Adventure) and VP of development for a consumer software and console game company. Since then, I’ve kept up with the changes in the game market and uses of games in education.

Creating these presentations push my thinking and my tech skills, and I’m always glad for that.

Sylvia

Games in Education – K12 Online Conference 2008

Now in its third year, the K12 Online Conference is underway. But don’t worry — you haven’t missed anything, it’s all online!

My session this year is Games in Education. The time limit was 20 minutes, so that was a challenge! To give it more depth, I also created a companion wiki page with lots more resources.

I cover topics including: Why games?; What’s wrong with edutainment?; Serious Games; Casual puzzle and logic games in the classroom; Virtual Worlds, Commercial off the Shelf (COTS) games; Overcoming Objections to using games in the classroom; and students designing and programming their own games. Yes, all in 20 min…

Beyond games, if you are interested in learning more about technology in education in general, and seeing presentations from educators all around the world, then you will find some great stuff on the K12 Online 2008 conference site. Since the conference is online and all the sessions are pre-recorded, they can be downloaded or viewed online anywhere, anytime. For first-timers, be sure to read the “how to” page here.

Check the conference blog for both recorded sessions and live chats with keynotes and presenters.

Sylvia

It’s been a while

So I’m back……. from Australia, and caught up with sleep and work (sort of).

Australia is an amazing country, with so many dedicated educators doing the right things for kids. I met lots of them, some old friends, and some new ones too!

I hope to put my ACEC presentation up on the blog soon – people seemed to like it. The paper is called, “Student-Centered Support Systems to Sustain Constructivist, Technology-rich Learning Environments”

Sylvia

You (yes, YOU) should present at a conference

Educational technology conferences happen all the time all over the country. There’s probably one going on every day of the year! At each of these conferences, the organizers look for interesting speakers and presentations that will excite the attendees. Where do these presenters come from? It’s no secret – THEY are YOU!

Steve Dembo of the Discovery Educator’s Network just wrote a blog post, You know you’re a rock star… Now prove it! that contains a handy list of all the state educational technology conferences and links to their session submission pages.

The problem is that many teachers don’t believe they are rock stars. I hear this all the time, ...oh, I’m just a teacher… lil ol’ me? ….. I just do my own thing in my classroom and no one even notices down the hall! Why would anyone else care?

Really now people, and especially you GenYES and TechYES teachers out there – you are doing some terrific, interesting things that other people want to hear about and understand better. You can’t fool me! I hear your stories all the time, and what you do is just as amazing as most conference sessions out there. The only difference is that the people presenting sessions have screwed up their courage and decided to share.

Putting forward good examples of student-centered learning and explaining how that happens is everyone’s job. Otherwise, people just assume it’s magic. So — why do you care about student empowerment? What does that look like? How do you create project-based experiences in this age of accountability? What is it you DO that creates the environment where students don’t just perform – they blossom?

Speaking at a conference is not “tooting your own horn”, it’s not egocentric, and it’s not just to get fame and fortune. Believe me, it’s not about fame and fortune! Actually, you can think of it as part of your professional development — it’s a balanced part of being an educator, doing reflective work about your own practice. OK, it’s scary to get up in front of people and talk, but for goodness sakes – you do this for a living! And how can a conference audience be any more scary than a gaggle of 14 year olds after gym class?

And now I’m going to up the stakes even more – take your students along on the adventure. If what you do in the classroom is meant to enable student voice and to empower students to own their own learning, let them take that experience outside the classroom. Let your students show others what it means to be a 21st century learner. Education conferences are becoming more open to student presenters, and really, it’s not that hard.

Sharing Student Voice at ConferencesTo help, I’ve written a how-to guide about taking students to conferences. It’s called, Sharing Student Voice: Students Presenting at Conferences. The focus is on making it an enriching experience for students AND also meeting the needs of the adult audience. It starts with how to propose sessions with students, goes through planning and preparation, and includes a handy Top 10 Tips for student presentations.

You can download the PDF here or read an online flipbook version.

And yes, I’m talking to YOU!

Sylvia