[iaoa-advisor] FOIS and CORE

Sun Apr 18 11:55:11 CEST 2021

Dear Laure,

Sorry for the delay.
Please find it in the attachment

best,
G

On Thu, Apr 15, 2021 at 7:45 PM Laure Vieu <vieu at irit.fr> wrote:

> Dear Giancarlo,
>
>
> Thanks for this.
>
> If you've got the attachment, could you please forward it?
> (I do not have access to the Google group linked)
>
>
> Best,
>
> Laure
>
>
> Le 15/04/2021 à 09:04, Guizzardi Giancarlo a écrit :
>
> Dear all,
>
> Some additional information on CORE
>
> best,
> Giancarlo
>
> Poviding additional info for CORE rankings
> <http://groups.google.com/a/core.edu.au/group/rankings/t/b93f7853f21ae669?utm_source=digest&utm_medium=email>
> Lin Padgham <lin.padgham at rmit.edu.au>: Apr 14 07:52PM +1000
>
> Dear All,
> For those wishing to supply additional information for review of the
> currently recommended changes to conferences in the CORE ranking, I attach
> pdfs of 2 sections of the full submission that was requested for new
> conferences, those wishing to be considered for upgrade, or those listed
> for review. These indicate some information which will be useful to the
> committee, particularly for those conferences that were evaluated on the
> basis of a comparator report only.
>
> I am currently investigating whether we can provide a web interface for
> submission of information (and if we are able to do that, this will be
> compulsory as it greatly assists in getting all the information into the DB
> and out to committees). I will post by the end of the week regarding how
> information should be provided. In the meantime these attachments provide
> some guidance as to what information will be of value.
>
> I do note that the section D2 where it asks for a list of at least 20*
> top *people
> in the area, and a report run through the WPP tool, is only of value if the
> list of people is chosen in a truly objective, repeatable and verifiable
> fashion, and is without implicit or explicit reference to the conference
> being evaluated. If these instructions are not followed, the report is of
> little value in this process.
>
> Similarly, in section C5, the list of top people regularly involved in the
> conference is only of value if some information is provided as to why you
> consider them a top person in the area, as well as their google scholar
> h-index. Just providing their university affiliation is of little value.
>
> Any additional supporting information can also be provided in any format
> desired. However it is important that information is independently
> verifiable data of some sort, and not simply opinions. While no single type
> of data is appropriate as a means of ranking, and a holistic view must be
> sought, there should always be some objective support for views that
> contradict what has been decided on the basis of data currently reviewed.
>
> If you are not planning to potentially provide further information but are
> just wanting to know why particular decisions were made, can i please ask
> that you wait until this information is uploaded to the DB along with the
> new rankings. At this stage there will be a more complete decision report,
> accounting for any extra information received. Also, it is overwhelming for
> me to try and respond to everyone, so I would greatly appreciate patience
> until the final reports are uploaded, except for cases where people feel
> that they may be able to contribute additional useful information.
>
> Regards,
> lin (CORE rankings coordinator)
>
>
>
> Lin Padgham
> Professor in Artificial Intelligence
> Computer Science, RMIT University
> Melbourne, Australia.
> ph. +61 3 9925 3214 (but email works better)
> ------------------------------
> *From:* Maria Keet <mkeet at cs.uct.ac.za> <mkeet at cs.uct.ac.za>
> *Sent:* Wednesday, April 14, 2021 4:00 PM
> *To:* Anthony Cohn <A.G.Cohn at leeds.ac.uk> <A.G.Cohn at leeds.ac.uk>;
> Guizzardi Giancarlo <Giancarlo.Guizzardi at unibz.it>
> <Giancarlo.Guizzardi at unibz.it>; Barry Smith <ifomis at gmail.com>
> <ifomis at gmail.com>; Laure Vieu <vieu at irit.fr> <vieu at irit.fr>
> *Cc:* Nicola Guarino <guarino at loa.istc.cnr.it> <guarino at loa.istc.cnr.it>;
> João Paulo Almeida <jpalmeida at ieee.org> <jpalmeida at ieee.org>; John
> Bateman <bateman at uni-bremen.de> <bateman at uni-bremen.de>; Stefano Borgo
> <stefano.borgo at cnr.it> <stefano.borgo at cnr.it>; Antony Galton
> <A.P.Galton at exeter.ac.uk> <A.P.Galton at exeter.ac.uk>; Janna Hastings
> <janna.hastings at gmail.com> <janna.hastings at gmail.com>; Heinrich Herre
> <heinrich.herre at imise.uni-leipzig.de>
> <heinrich.herre at imise.uni-leipzig.de>; Werner Kuhn <werner.kuhn at gmail.com>
> <werner.kuhn at gmail.com>; Riichiro Mizoguchi <mizo at jaist.ac.jp>
> <mizo at jaist.ac.jp>; Mark Musen <musen at stanford.edu> <musen at stanford.edu>;
> Leo Obrst <lobrst at gmail.com> <lobrst at gmail.com>; Barry Smith
> <phismith at buffalo.edu> <phismith at buffalo.edu>; Zena Wood
> <Z.M.Wood2 at exeter.ac.uk> <Z.M.Wood2 at exeter.ac.uk>; Roberta Ferrario
> <roberta.ferrario at loa.istc.cnr.it> <roberta.ferrario at loa.istc.cnr.it>;
> Michael Gruninger <gruninger at mie.utoronto.ca> <gruninger at mie.utoronto.ca>;
> thomas.studer at inf.unibe.ch <thomas.studer at inf.unibe.ch>
> <thomas.studer at inf.unibe.ch>; IAOA Executive Council
> <iaoa-council at ovgu.de> <iaoa-council at ovgu.de>
> *Subject:* Re: FOIS and CORE
>
> Dear Tony, Giancarlo, All,
>
> On 14/04/2021 15:45, Anthony Cohn wrote:
>
> An interesting discussion and of course I agree  it’s worth making a case
> against the “demotion”.  I agree about the h-index point – scopus has a
> notion of “field weighted impact factor” – which adjusts for the size of
> the community. I don’t know if scopus computes this for conferences, but
> given the relatively small size of our community this might be an argument
> to make?
>
> agreed, hence the "a need to spin that story better" note in my email.
> afaik, it is something that they consider, but I'm not privy to all those
> details.
> and indeed, it's people beyond the authors' standings and their repeat
> participation, including those of the event organisers and participants,
> among others.
> but as long as we don't know the reasons by it got downgraded, it's
> guesswork why exactly
>
> Regards,
> Maria
>
>
>
> Best wishes      Tony
>
>
>
> *From:* Guizzardi Giancarlo <Giancarlo.Guizzardi at unibz.it>
> <Giancarlo.Guizzardi at unibz.it>
> *Sent:* 14 April 2021 14:26
> *To:* Maria Keet <mkeet at cs.uct.ac.za> <mkeet at cs.uct.ac.za>; Barry Smith
> <ifomis at gmail.com> <ifomis at gmail.com>; Laure Vieu <vieu at irit.fr>
> <vieu at irit.fr>
> *Cc:* Nicola Guarino <guarino at loa.istc.cnr.it> <guarino at loa.istc.cnr.it>;
> João Paulo Almeida <jpalmeida at ieee.org> <jpalmeida at ieee.org>; John
> Bateman <bateman at uni-bremen.de> <bateman at uni-bremen.de>; Stefano Borgo
> <stefano.borgo at cnr.it> <stefano.borgo at cnr.it>; Anthony Cohn
> <A.G.Cohn at leeds.ac.uk> <A.G.Cohn at leeds.ac.uk>; Antony Galton
> <A.P.Galton at exeter.ac.uk> <A.P.Galton at exeter.ac.uk>; Janna Hastings
> <janna.hastings at gmail.com> <janna.hastings at gmail.com>; Heinrich Herre
> <heinrich.herre at imise.uni-leipzig.de>
> <heinrich.herre at imise.uni-leipzig.de>; Werner Kuhn <werner.kuhn at gmail.com>
> <werner.kuhn at gmail.com>; Riichiro Mizoguchi <mizo at jaist.ac.jp>
> <mizo at jaist.ac.jp>; Mark Musen <musen at stanford.edu> <musen at stanford.edu>;
> Leo Obrst <lobrst at gmail.com> <lobrst at gmail.com>; Barry Smith
> <phismith at buffalo.edu> <phismith at buffalo.edu>; Zena Wood
> <Z.M.Wood2 at exeter.ac.uk> <Z.M.Wood2 at exeter.ac.uk>; Roberta Ferrario
> <roberta.ferrario at loa.istc.cnr.it> <roberta.ferrario at loa.istc.cnr.it>;
> Michael Gruninger <gruninger at mie.utoronto.ca> <gruninger at mie.utoronto.ca>;
> thomas.studer at inf.unibe.ch; IAOA Executive Council <iaoa-council at ovgu.de>
> <iaoa-council at ovgu.de>
> *Subject:* Re: FOIS and CORE
>
>
>
> Dear Maria and all,
>
>
>
> CORE does periodic revision in its rankings from time to time.
>
>
>
> They have already requested input from the different communities many
> months ago.
>
> As a community, we missed that. I wasn't concerned about that because the
> conference was classified as A
>
> (well, in theory, we could have tried to make the case for A* but that
> would not have worked out as
>
> our demotion shows...).
>
>
>
> The criteria used by CORE is a mixture of objective and subjective points
> (again, the communities
>
> try to make the case considering both types of indicators). FOIS does not
> do well h-index-wise but
>
> that is mainly due to the fact that it is a biannual conference (I won't
> even enter the discussion
>
> of how much of a bad idea is to use h-index to judge conferences...). But
> that is the same for KR
>
> and KR managed to revise their initially bad evaluation. Again, this
> requires an active community effort.
>
> Other conferences that are small but prestigious in their relevant
> communities have
>
> managed to even talk their way up to A* (e.g., PODS). ER is trying the
> same move.
>
>
>
> As for subjective aspects, some of the points that are taken very
> seriously is the impact
>
> and scientific stature of people playing key roles in the conference (PC
> chairs and members,
>
> general chairs, keynote speakers, frequent authors, etc.).
>
>
>
> I think we should react now by contacting them and defending the
> importance of the event
>
> as the most important event for this community and highlighting some of
> these points
>
>
>
> best,
>
> Giancarlo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ------------------------------
>
> *From:* Maria Keet <mkeet at cs.uct.ac.za>
> *Sent:* Wednesday, April 14, 2021 10:33 AM
> *To:* Guizzardi Giancarlo <Giancarlo.Guizzardi at unibz.it>; Barry Smith <
> ifomis at gmail.com>; Laure Vieu <vieu at irit.fr>
> *Cc:* Nicola Guarino <guarino at loa.istc.cnr.it>; João Paulo Almeida <
> jpalmeida at ieee.org>; John Bateman <bateman at uni-bremen.de>; Stefano Borgo <
> stefano.borgo at cnr.it>; Cohn <A.G.Cohn at leeds.ac.uk>; Antony Galton <
> A.P.Galton at exeter.ac.uk>; Janna Hastings <janna.hastings at gmail.com>;
> Heinrich Herre <heinrich.herre at imise.uni-leipzig.de>; Werner Kuhn <
> werner.kuhn at gmail.com>; Riichiro Mizoguchi <mizo at jaist.ac.jp>; Mark Musen
> <musen at stanford.edu>; Leo Obrst <lobrst at gmail.com>; Barry Smith <
> phismith at buffalo.edu>; Zena Wood <Z.M.Wood2 at exeter.ac.uk>; Roberta
> Ferrario <roberta.ferrario at loa.istc.cnr.it>; Michael Gruninger <
> gruninger at mie.utoronto.ca>; thomas.studer at inf.unibe.ch <
> thomas.studer at inf.unibe.ch>; IAOA Executive Council <iaoa-council at ovgu.de>
> *Subject:* Re: FOIS and CORE
>
>
>
> Dear Giancarlo, All,
>
> It may be of use to first find out from them why the re-evaluated it
> differently.
> Good quality papers isn't the only criterion they use. It's also, e.g.,
> high performers among the authors, but I assume we pass that as well. And
> then there's the h-index of the conference, which doesn't do well, as if
> FOIS doesn't really have any impact, actually. Well, anyway, that's what it
> looked like when EKAW was putting the material together, when Enrico Motta
> was showing off his tool for computing citation metrics during the EKAW
> steering committee meeting. Anyway, if I had to put in a bet for reason
> why, it would be that and a need to spin that story better. Still, to be
> sure, perhaps the EC can try to find out why from Lin first?
>
> Regards,
> Maria
>
>
> ----
>
> Dr. Maria Keet
>
> Associate Professor
>
> Department of Computer Science
>
> University of Cape Town
>
> Cape Town, South Africa
>
> tel: +27 21 650 2667
>
> fax: +27 21 650 3551
>
> email: mkeet at cs.uct.ac.za
>
> work: http://www.cs.uct.ac.za
>
> home: http://www.meteck.org
>
>
>
> On 13/04/2021 21:44, Guizzardi Giancarlo wrote:
>
> Dear all,
>
>
>
> Let me raise another concern now.
>
> Please see the message below from the CORE committee,
>
> in which they are revising the classification of many CS conferences.
>
>
> As many of you know, CORE is important for computer science faculties.
>
> Not only directly but also because they build up into several national
> evaluation systems.
>
> If the classification of a conference goes down, people will prefer to
> send their
>
> best papers somewhere else.
>
>
>
> According to this revised classification, FOIS went from an A conference
> to a B conference.
>
> I think there is room there for protesting this classification before
> their final decisions
>
> but we need to move fast and institutionally, i.e., IAOA should do it.
>
>
>
> Best regards,
>
> Giancarlo
>
>
>
> CORE conference rankings (preliminary) changes
> <http://groups.google.com/a/core.edu.au/group/rankings/t/9f437bd0cd9470bf?utm_source=digest&utm_medium=email>
>
> Lin Padgham <lin.padgham at rmit.edu.au>: Apr 11 11:06AM +1000
>
> Dear CS colleagues,
> The CORE committees have now finished reviewing the approximately 400 new
> and existing conferences that were part of this review round.
> More than 50% of the conferences reviewed retained their existing rank.
> Consistent with our previous practice, we publish here
> <
> https://docs.google.com/spreadsheets/d/17cnG1Vgyjdu3pGdIHrvyw6HCithPAvTeDd_YB_FVOK0/edit?usp=sharing
> >
> a list of all planned ranking changes, allowing a short period for any
> additional information if community members consider there has been an
> error of judgement.* If you wish to potentially question a ranking change,
> please notify your intent with an email to lin.padgham at gmail.com
> <lin.padgham at gmail.com>, with subject "Rankings <confname>" by April
> 18th.*
> You will then be provided with the submission and the detailed decision
> report, referencing the data on which the decision was based. *If you
> consider you have additional data which may change the committee's
> recommendation, this must be provided by April 25th*. Committees will then
> review this, prior to finalisation and upload of the new CORE conference
> ranking.
> Regards,
> Lin (CORE Rankings Co-ordinator)
>
> Lin Padgham
> Professor in Artificial Intelligence
> Computer Science, RMIT University
> Melbourne, Australia.
> ph. +61 3 9925 3214 (but email works better)
> (please note I only work Wednesdays)
>
>
>
>
>
>
>
>
>
> _________________________________________________________________________
> Msg Archives: https://listserv.ovgu.de/pipermail/iaoa-advisor/
> IAOA wiki:    http://ontolog-02.cim3.net/wiki/IAOA
> IAOA website: http://iaoa.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.ovgu.de/pipermail/iaoa-advisor/attachments/20210418/dbe191d3/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sectionC(1).pdf
Type: application/pdf
Size: 694489 bytes
Desc: not available
URL: <http://listserv.ovgu.de/pipermail/iaoa-advisor/attachments/20210418/dbe191d3/attachment-0002.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sectionD.pdf
Type: application/pdf
Size: 387371 bytes
Desc: not available
URL: <http://listserv.ovgu.de/pipermail/iaoa-advisor/attachments/20210418/dbe191d3/attachment-0003.pdf>