Thank you FQXi, but I have concerns ...
I also made some comments at the forum for discussing the contest in general. I and some other commenters expressed some disappointment about there not being award/s for non-professional or amateur submitters (a possibility expressed in the rules for the contest.) However, my concerns go well beyond those issues, as seen below. So, tell me what you think, suggestions, etc. (I am trying to separate discussion of the scientific issues from Essay Contest issues, but I realize they might intermingle: especially since one of my concerns was the inadequate credit given papers - like mine - that proposed experimental tests, the few of them standing out "like sore thumbs" in a field of arm-chair fancy.
Brendan Foster and colleagues,
Dear Brendan, all;
I sadly write to you as a disappointed contributor, with a posted essay in your contest "Is Reality Digital or Analog." ( http://fqxi.org/community/
First, I applaud the concept and mission of FQXi, and welcome the opportunity in principle offered by your contests. I applaud your openness to amateurs and semi-professionals lacking institutional support, and your giving a forum to challenging ideas and to those who produce good work but have trouble pushing submissions through the journal refereeing process, etc. What disappoints me is how this contest turned out so far (I haven't time to study previous contests for comparison.) Most relevant, perplexing and saddening to me, is the prevalence of low community ratings. There is something wrong when most essays in such a contest rate 3-5, and the "best" essay pegs a flabby 5.4! (On the Depth of Quantum Space by Daniele Oriti 76 posts ... Feb. 16, 2011 @ 14:23 GMT Community Rating: 5.4 (34 ratings) Public Rating: 6.0 (22 ratings)) That's like an "F" grade. What are the community raters looking for, that they didn't find?
Note that since essay submitters become community raters, and relative rankings determine success, then the least responsible CRs sadly have a conflict of interest: to give low ratings to other submissions that may be a threat, or that they dislike intuitively regardless of merit, etc. At least we should see separate ratings from fellow authors, compared to standing FQXi members. I don't even know, did the former pull down the latter's ratings, or vice versa? If the authors: why should their judgment as raters be any better than the papers that got low average ratings? Perhaps it would be better for applicants to just not rate each other except as public raters. Their comments can show what the consensus is.
My situation: instead of offering a new theory or challenge to accepted physics, I critiqued a popular alternative to traditional quantum theory. I also proposed a way to empirically test that interpretation. Few essays offered practical tests of any theory, most were very speculative. I received many supportive comments, and some from significant figures in physics like T. Dorigo. Not all commenters agreed with my thesis, yet none offered a correction to my technical points, my line of reasoning, said I did a poor job, or rebutted my claims as to the experimental outcome (even if not convinced of its implications), etc. Even so, my community rating was a mere 3.6 (Public = 6.5.) Why didn't any of the low raters explain what bothered them? Isn't that part of their job, formally or informally as the case may be? Most raters should at least have been impressed by someone proposing an eminently practical experimental test of what has just been a philosophical argument.
In any case, ratings need to be justified, not just a click and forget. I can see that applicant raters need to be anonymous to prevent retaliation, etc. However, perhaps standing FQXi members should (as in Olympic competition etc.) openly give grades and explain their ratings. The current system just does not work since raters are apparently too swayed by impulse attractions or distastes (after all, the decoherence theory I critiqued is a popular way to evade measurement paradoxes.) I realize that the expert judges will add further and could shift ratings around, but I can't see at FQXi website what they have accomplished. Also, the rules say the EJs will be "peers" of the applicants - how will you find "peers" for professed or presumed amateurs? Furthermore, if EJs don't revisit promising papers not among the finalists, what valuable work might be missed that wasn't appreciated in the community ratings? Is there a way to appeal to someone, "look at this again"?
Also: your contest has a category for "non-professional and/or non-academic," yet you don't seem to have a clear-cut process for identifying amateurs. I see no defined list. Is it assumed from the submitter not providing a professional affiliation? Isn't that subject to misinterpretation? I stated in my "Author Bio" that I count as an amateur (note: the term is defined in social context) and am not aware of any other posted submitter saying so directly in their Bio. (I did not make thorough check and some implied, or in comments.) Furthermore, if amateurs are a special group does that mean such a submitter still has a chance of that recognition, even if not among the general "finalists"?
Others have expressed similar concerns to mine here and there. See for example critique regarding amateur status, public ratings etc. in comment to my essay, "basudeba wrote on Mar. 20, 2011 @ 06:14 GMT." This author/s may have contacted you already. (I set aside my concerns about public ratings, since e.g. there are common criticisms.) Please distribute my comments widely, and among FQXi members, the applicants, judges etc. I want my concerns to be considered and responded to. I hope FQXi will post my letter so that everyone involved can see what sorts of concerns entrants have. Perhaps it can start a post for discussion. There is a valid case that although FQXi essay contests are a helpful effort in principle, their manner of administration needs clarification and reform. In any case I continue to support your trailblazing and egalitarian mission.
PS: I plan to enter this letter as a comment to my essay. If you object, please tell me and explain. Thank you for your interest and concern.
Labels: contests publishing politics