From vieu at irit.fr Wed Apr 14 10:41:58 2021 From: vieu at irit.fr (Laure Vieu) Date: Wed, 14 Apr 2021 10:41:58 +0200 Subject: [iaoa-advisor] FOIS and CORE In-Reply-To: <7990605e-b8fd-0fa8-2eb5-641757be7bfb@cs.uct.ac.za> References: <8713385f-0b01-7af7-f428-09dfa93d21a7@irit.fr> <7990605e-b8fd-0fa8-2eb5-641757be7bfb@cs.uct.ac.za> Message-ID: <8c877588-3a0c-354e-afe1-ff7d7a545697@irit.fr> An HTML attachment was scrubbed... URL: From vieu at irit.fr Wed Apr 14 16:07:03 2021 From: vieu at irit.fr (Laure Vieu) Date: Wed, 14 Apr 2021 16:07:03 +0200 Subject: [iaoa-advisor] FOIS and CORE In-Reply-To: <01ac38d7-3896-7e6b-d197-af7d05c32c48@cs.uct.ac.za> References: <8713385f-0b01-7af7-f428-09dfa93d21a7@irit.fr> <7990605e-b8fd-0fa8-2eb5-641757be7bfb@cs.uct.ac.za> <01ac38d7-3896-7e6b-d197-af7d05c32c48@cs.uct.ac.za> Message-ID: An HTML attachment was scrubbed... URL: From vieu at irit.fr Thu Apr 15 11:10:24 2021 From: vieu at irit.fr (Laure Vieu) Date: Thu, 15 Apr 2021 11:10:24 +0200 Subject: [iaoa-advisor] Your advice on the Proposed Elimination of Membership Fees for IAOA In-Reply-To: References: <8713385f-0b01-7af7-f428-09dfa93d21a7@irit.fr> Message-ID: Dear Anthony and Maria, dear all, We have already thoroughly examined the two grades of membership solution within the EC. It has many drawbacks, among which, importantly, requiring to change our statutes which do not right now allow for two classes of members and, obviously, creating a division among the members in a more conspicuous and official way than the original proposal to grant access to those who donate enough, which has been criticized already during the public discussion for creating second-class citizens. For the future funding scheme, as Fabian explained, we do not currently have high regular expenses (of course, buying a large package access to IOS Press could change this, this is why a special fund is an interesting idea), and in the past, the bulk of the money gathered has been essentially from conferences surpluses. We will continue to enforce the policy that IAOA conferences should run in a fully autonomous financial mode, covering also student grants etc, and that IAOA contributes with advance money and as insurance against major events. Hopefully, surpluses will continue to come in once in a while. And hopefully, we will improve our ability to raise donations. Note that since institutions can still join for a fee, those who are in position to use institutional funds may join as institution instead of giving a donation (which often is more problematic institutionally). Finally, we plan to assess regularly the sustainability of the new model, and might propose to change it again. Best, Laure Le 15/04/2021 à 09:25, Maria Keet a écrit : > Hi Anthony, > > On 14/04/2021 20:02, Galton, Antony wrote: >> We could have two grades of membership - full membership which >> requires a fee but gives free access to the IOS publications as at >> present, and something like associate membership which has no fee as >> per the current proposal. Or would this stigmatize the latter as >> second-class citizens? >> >> The main problem with abolishing membership fees entirely, as I see >> it, is: How will IAOA be funded in future? Is there sufficient income >> from sources other than membership fees to sustain the activities of >> the organisation? It would be very risky to rely on donations to make >> up any shortfall. > there's a huge amount of money in the account to last for a while (and > weather a major event disaster), and surplusses of IAOA's main events > go to the IAOA account, so it'd be basically 'giving back' to the > community in kind what we've all contributed over a while already. > > Regards, > Maria >> >> Antony >> >> ________________________________________ >> From: Laure Vieu >> Sent: 13 April 2021 19:01 >> To: Nicola Guarino; 'João Paulo Almeida'; John Bateman; Stefano >> Borgo; Cohn; Galton, Antony; Giancarlo Guizzardi; Janna Hastings; >> Heinrich Herre; Werner Kuhn; Riichiro Mizoguchi; Mark Musen; Leo >> Obrst; Barry Smith; Wood, Zena; Roberta Ferrario; Michael Gruninger; >> Maria Keet; thomas.studer at inf.unibe.ch >> Cc: IAOA Executive Council >> Subject: Your advice on the Proposed Elimination of Membership Fees >> for IAOA >> >> CAUTION: This email originated from outside of the organisation. Do >> not click links or open attachments unless you recognise the sender >> and know the content is safe. >> >> >> Dear IAOA Advisory Board members, >> >> We highly value your advisory role on important matters regarding our >> association. >> >> As you probably have noticed, a discussion on a proposal made by the >> Executive Council (EC) is currently going on the IAOA-member mailing >> list, taking advantage of the Assembly period. (See below the >> original message with the proposal.) >> We believe this proposal, the complete elimination of membership >> fees, has the potential to bring an important change in our >> association. It is a proposal, not a decision, and if adopted, it >> still needs to be refined or altered in various ways. Several ideas >> in this respect have already been put forward during the discussion. >> >> Therefore, in the name of the whole EC, I would like to warmly >> encourage you to share your views, taking part to the public >> discussion going on IAOA-member mailing list >> (iaoa-member at ovgu.de), and help us >> reaching the best decision.  Some of you have already contributed to >> this discussion, for which we are very grateful. >> Alternatively, you may send me a message that I will share with the >> whole EC. >> >> Best regards, >> >> Laure Vieu >> IAOA president >> >> >> -------- Message transféré -------- >> Sujet :         [iaoa-member] Proposed Elimination of Membership Fees >> -- To be discussed during the current Assembly, till April 30 >> Date :  Tue, 6 Apr 2021 18:46:45 +0200 >> De :    Laure Vieu >> Répondre à :    [IAOA-member] >> >> Pour :  iaoa-member at ovgu.de >> >> >> >> Dear IAOA member, >> >> We would like to take advantage of this year’s assembly to share a >> proposal by the Executive Council to eliminate all required >> membership fees, for your discussion and feedback. The motivation >> behind this is to decrease any financial barriers to membership, and >> so make the association more open and accessible to a wider ontology >> community. This move will also remove any inconvenience associated >> with annual membership renewals -- once registered under the new >> framework, members will remain so unless they choose to close their >> membership. That said, we are hoping for donations from members to >> fund IAOA activities in the future. >> >> Under this new scheme, IAOA members will no longer automatically >> receive free access to those IOS Press publications that currently >> are open to IAOA members. It’s been suggested that any donation above >> a certain amount may receive free access to IOS Press publications >> for one calendar year after the date of the donation, however the >> details of this remain to be determined. Regardless, all current, >> paid members would continue to receive access to IOS Press >> publications until the date of their (paid) membership expiry. >> >> We are proposing the adoption of this approach on a trial basis. If >> implemented, the idea would be to monitor the impact for any >> unforeseen consequences, modifying the approach or reverting to the >> original fees scheme if problems do arise. That said, we do not >> anticipate that this change will impact the regular activities of the >> IAOA. >> >> Please let us, and all, know what you think of the idea. >> The discussion on this topic is open till the end of the current >> assembly, on April 30, through this mailing list. >> >> Laure Vieu, IAOA President, and the whole IAOA Executive Council >> >> >> > From A.G.Cohn at leeds.ac.uk Thu Apr 15 11:25:11 2021 From: A.G.Cohn at leeds.ac.uk (Anthony Cohn) Date: Thu, 15 Apr 2021 09:25:11 +0000 Subject: [iaoa-advisor] Your advice on the Proposed Elimination of Membership Fees for IAOA In-Reply-To: References: <8713385f-0b01-7af7-f428-09dfa93d21a7@irit.fr> Message-ID: Dear Laure Thanks for this and for reminding me about the institutional membership which addresses the problem I was trying to solve! Best wishes Tony > -----Original Message----- > From: iaoa-advisor On Behalf Of Laure Vieu > Sent: 15 April 2021 10:10 > To: iaoa-advisor at ovgu.de > Cc: IAOA Executive Council > Subject: Re: [iaoa-advisor] Your advice on the Proposed Elimination of > Membership Fees for IAOA > > Dear Anthony and Maria, dear all, > > We have already thoroughly examined the two grades of membership solution > within the EC. It has many drawbacks, among which, importantly, requiring to > change our statutes which do not right now allow for two classes of members > and, obviously, creating a division among the members in a more conspicuous > and official way than the original proposal to grant access to those who > donate enough, which has been criticized already during the public discussion > for creating second-class citizens. > > For the future funding scheme, as Fabian explained, we do not currently have > high regular expenses (of course, buying a large package access to IOS Press > could change this, this is why a special fund is an interesting idea), and in the > past, the bulk of the money gathered has been essentially from conferences > surpluses. > We will continue to enforce the policy that IAOA conferences should run in a > fully autonomous financial mode, covering also student grants etc, and that > IAOA contributes with advance money and as insurance against major events. > Hopefully, surpluses will continue to come in once in a while. And hopefully, > we will improve our ability to raise donations. > Note that since institutions can still join for a fee, those who are in position to > use institutional funds may join as institution instead of giving a donation > (which often is more problematic institutionally). > > Finally, we plan to assess regularly the sustainability of the new model, and > might propose to change it again. > > Best, > > Laure > > > Le 15/04/2021 à 09:25, Maria Keet a écrit : > > Hi Anthony, > > > > On 14/04/2021 20:02, Galton, Antony wrote: > >> We could have two grades of membership - full membership which > >> requires a fee but gives free access to the IOS publications as at > >> present, and something like associate membership which has no fee as > >> per the current proposal. Or would this stigmatize the latter as > >> second-class citizens? > >> > >> The main problem with abolishing membership fees entirely, as I see > >> it, is: How will IAOA be funded in future? Is there sufficient income > >> from sources other than membership fees to sustain the activities of > >> the organisation? It would be very risky to rely on donations to make > >> up any shortfall. > > there's a huge amount of money in the account to last for a while (and > > weather a major event disaster), and surplusses of IAOA's main events > > go to the IAOA account, so it'd be basically 'giving back' to the > > community in kind what we've all contributed over a while already. > > > > Regards, > > Maria > >> > >> Antony > >> > >> ________________________________________ > >> From: Laure Vieu > >> Sent: 13 April 2021 19:01 > >> To: Nicola Guarino; 'João Paulo Almeida'; John Bateman; Stefano > >> Borgo; Cohn; Galton, Antony; Giancarlo Guizzardi; Janna Hastings; > >> Heinrich Herre; Werner Kuhn; Riichiro Mizoguchi; Mark Musen; Leo > >> Obrst; Barry Smith; Wood, Zena; Roberta Ferrario; Michael Gruninger; > >> Maria Keet; thomas.studer at inf.unibe.ch > >> Cc: IAOA Executive Council > >> Subject: Your advice on the Proposed Elimination of Membership Fees > >> for IAOA > >> > >> CAUTION: This email originated from outside of the organisation. Do > >> not click links or open attachments unless you recognise the sender > >> and know the content is safe. > >> > >> > >> Dear IAOA Advisory Board members, > >> > >> We highly value your advisory role on important matters regarding our > >> association. > >> > >> As you probably have noticed, a discussion on a proposal made by the > >> Executive Council (EC) is currently going on the IAOA-member mailing > >> list, taking advantage of the Assembly period. (See below the > >> original message with the proposal.) We believe this proposal, the > >> complete elimination of membership fees, has the potential to bring > >> an important change in our association. It is a proposal, not a > >> decision, and if adopted, it still needs to be refined or altered in > >> various ways. Several ideas in this respect have already been put > >> forward during the discussion. > >> > >> Therefore, in the name of the whole EC, I would like to warmly > >> encourage you to share your views, taking part to the public > >> discussion going on IAOA-member mailing list > >> (iaoa-member at ovgu.de), and help us > >> reaching the best decision.  Some of you have already contributed to > >> this discussion, for which we are very grateful. > >> Alternatively, you may send me a message that I will share with the > >> whole EC. > >> > >> Best regards, > >> > >> Laure Vieu > >> IAOA president > >> > >> > >> -------- Message transféré -------- > >> Sujet :         [iaoa-member] Proposed Elimination of Membership Fees > >> -- To be discussed during the current Assembly, till April 30 Date : > >> Tue, 6 Apr 2021 18:46:45 +0200 De :    Laure Vieu > >> > >> Répondre à :    [IAOA-member] > >> > >> Pour :  iaoa-member at ovgu.de > >> > >> > >> > >> Dear IAOA member, > >> > >> We would like to take advantage of this year’s assembly to share a > >> proposal by the Executive Council to eliminate all required > >> membership fees, for your discussion and feedback. The motivation > >> behind this is to decrease any financial barriers to membership, and > >> so make the association more open and accessible to a wider ontology > >> community. This move will also remove any inconvenience associated > >> with annual membership renewals -- once registered under the new > >> framework, members will remain so unless they choose to close their > >> membership. That said, we are hoping for donations from members to > >> fund IAOA activities in the future. > >> > >> Under this new scheme, IAOA members will no longer automatically > >> receive free access to those IOS Press publications that currently > >> are open to IAOA members. It’s been suggested that any donation above > >> a certain amount may receive free access to IOS Press publications > >> for one calendar year after the date of the donation, however the > >> details of this remain to be determined. Regardless, all current, > >> paid members would continue to receive access to IOS Press > >> publications until the date of their (paid) membership expiry. > >> > >> We are proposing the adoption of this approach on a trial basis. If > >> implemented, the idea would be to monitor the impact for any > >> unforeseen consequences, modifying the approach or reverting to the > >> original fees scheme if problems do arise. That said, we do not > >> anticipate that this change will impact the regular activities of the > >> IAOA. > >> > >> Please let us, and all, know what you think of the idea. > >> The discussion on this topic is open till the end of the current > >> assembly, on April 30, through this mailing list. > >> > >> Laure Vieu, IAOA President, and the whole IAOA Executive Council > >> > >> > >> > > > > __________________________________________________________________ > _______ > Msg Archives: https://listserv.ovgu.de/pipermail/iaoa-advisor/ > IAOA wiki: http://ontolog-02.cim3.net/wiki/IAOA > IAOA website: http://iaoa.org From vieu at irit.fr Thu Apr 15 19:45:00 2021 From: vieu at irit.fr (Laure Vieu) Date: Thu, 15 Apr 2021 19:45:00 +0200 Subject: [iaoa-advisor] FOIS and CORE In-Reply-To: References: <8713385f-0b01-7af7-f428-09dfa93d21a7@irit.fr> <7990605e-b8fd-0fa8-2eb5-641757be7bfb@cs.uct.ac.za> <01ac38d7-3896-7e6b-d197-af7d05c32c48@cs.uct.ac.za> Message-ID: An HTML attachment was scrubbed... URL: From gguizzardi at gmail.com Sun Apr 18 11:55:11 2021 From: gguizzardi at gmail.com (Giancarlo Guizzardi) Date: Sun, 18 Apr 2021 11:55:11 +0200 Subject: [iaoa-advisor] FOIS and CORE In-Reply-To: References: <8713385f-0b01-7af7-f428-09dfa93d21a7@irit.fr> <7990605e-b8fd-0fa8-2eb5-641757be7bfb@cs.uct.ac.za> <01ac38d7-3896-7e6b-d197-af7d05c32c48@cs.uct.ac.za> Message-ID: Dear Laure, Sorry for the delay. Please find it in the attachment best, G On Thu, Apr 15, 2021 at 7:45 PM Laure Vieu wrote: > Dear Giancarlo, > > > Thanks for this. > > If you've got the attachment, could you please forward it? > (I do not have access to the Google group linked) > > > Best, > > Laure > > > Le 15/04/2021 à 09:04, Guizzardi Giancarlo a écrit : > > Dear all, > > Some additional information on CORE > > best, > Giancarlo > > Poviding additional info for CORE rankings > > Lin Padgham : Apr 14 07:52PM +1000 > > Dear All, > For those wishing to supply additional information for review of the > currently recommended changes to conferences in the CORE ranking, I attach > pdfs of 2 sections of the full submission that was requested for new > conferences, those wishing to be considered for upgrade, or those listed > for review. These indicate some information which will be useful to the > committee, particularly for those conferences that were evaluated on the > basis of a comparator report only. > > I am currently investigating whether we can provide a web interface for > submission of information (and if we are able to do that, this will be > compulsory as it greatly assists in getting all the information into the DB > and out to committees). I will post by the end of the week regarding how > information should be provided. In the meantime these attachments provide > some guidance as to what information will be of value. > > I do note that the section D2 where it asks for a list of at least 20* > top *people > in the area, and a report run through the WPP tool, is only of value if the > list of people is chosen in a truly objective, repeatable and verifiable > fashion, and is without implicit or explicit reference to the conference > being evaluated. If these instructions are not followed, the report is of > little value in this process. > > Similarly, in section C5, the list of top people regularly involved in the > conference is only of value if some information is provided as to why you > consider them a top person in the area, as well as their google scholar > h-index. Just providing their university affiliation is of little value. > > Any additional supporting information can also be provided in any format > desired. However it is important that information is independently > verifiable data of some sort, and not simply opinions. While no single type > of data is appropriate as a means of ranking, and a holistic view must be > sought, there should always be some objective support for views that > contradict what has been decided on the basis of data currently reviewed. > > If you are not planning to potentially provide further information but are > just wanting to know why particular decisions were made, can i please ask > that you wait until this information is uploaded to the DB along with the > new rankings. At this stage there will be a more complete decision report, > accounting for any extra information received. Also, it is overwhelming for > me to try and respond to everyone, so I would greatly appreciate patience > until the final reports are uploaded, except for cases where people feel > that they may be able to contribute additional useful information. > > Regards, > lin (CORE rankings coordinator) > > > > Lin Padgham > Professor in Artificial Intelligence > Computer Science, RMIT University > Melbourne, Australia. > ph. +61 3 9925 3214 (but email works better) > ------------------------------ > *From:* Maria Keet > *Sent:* Wednesday, April 14, 2021 4:00 PM > *To:* Anthony Cohn ; > Guizzardi Giancarlo > ; Barry Smith > ; Laure Vieu > *Cc:* Nicola Guarino ; > João Paulo Almeida ; John > Bateman ; Stefano Borgo > ; Antony Galton > ; Janna Hastings > ; Heinrich Herre > > ; Werner Kuhn > ; Riichiro Mizoguchi > ; Mark Musen ; > Leo Obrst ; Barry Smith > ; Zena Wood > ; Roberta Ferrario > ; > Michael Gruninger ; > thomas.studer at inf.unibe.ch > ; IAOA Executive Council > > *Subject:* Re: FOIS and CORE > > Dear Tony, Giancarlo, All, > > On 14/04/2021 15:45, Anthony Cohn wrote: > > An interesting discussion and of course I agree it’s worth making a case > against the “demotion”. I agree about the h-index point – scopus has a > notion of “field weighted impact factor” – which adjusts for the size of > the community. I don’t know if scopus computes this for conferences, but > given the relatively small size of our community this might be an argument > to make? > > agreed, hence the "a need to spin that story better" note in my email. > afaik, it is something that they consider, but I'm not privy to all those > details. > and indeed, it's people beyond the authors' standings and their repeat > participation, including those of the event organisers and participants, > among others. > but as long as we don't know the reasons by it got downgraded, it's > guesswork why exactly > > Regards, > Maria > > > > Best wishes Tony > > > > *From:* Guizzardi Giancarlo > > *Sent:* 14 April 2021 14:26 > *To:* Maria Keet ; Barry Smith > ; Laure Vieu > > *Cc:* Nicola Guarino ; > João Paulo Almeida ; John > Bateman ; Stefano Borgo > ; Anthony Cohn > ; Antony Galton > ; Janna Hastings > ; Heinrich Herre > > ; Werner Kuhn > ; Riichiro Mizoguchi > ; Mark Musen ; > Leo Obrst ; Barry Smith > ; Zena Wood > ; Roberta Ferrario > ; > Michael Gruninger ; > thomas.studer at inf.unibe.ch; IAOA Executive Council > > *Subject:* Re: FOIS and CORE > > > > Dear Maria and all, > > > > CORE does periodic revision in its rankings from time to time. > > > > They have already requested input from the different communities many > months ago. > > As a community, we missed that. I wasn't concerned about that because the > conference was classified as A > > (well, in theory, we could have tried to make the case for A* but that > would not have worked out as > > our demotion shows...). > > > > The criteria used by CORE is a mixture of objective and subjective points > (again, the communities > > try to make the case considering both types of indicators). FOIS does not > do well h-index-wise but > > that is mainly due to the fact that it is a biannual conference (I won't > even enter the discussion > > of how much of a bad idea is to use h-index to judge conferences...). But > that is the same for KR > > and KR managed to revise their initially bad evaluation. Again, this > requires an active community effort. > > Other conferences that are small but prestigious in their relevant > communities have > > managed to even talk their way up to A* (e.g., PODS). ER is trying the > same move. > > > > As for subjective aspects, some of the points that are taken very > seriously is the impact > > and scientific stature of people playing key roles in the conference (PC > chairs and members, > > general chairs, keynote speakers, frequent authors, etc.). > > > > I think we should react now by contacting them and defending the > importance of the event > > as the most important event for this community and highlighting some of > these points > > > > best, > > Giancarlo > > > > > > > > > > > > > > > > > ------------------------------ > > *From:* Maria Keet > *Sent:* Wednesday, April 14, 2021 10:33 AM > *To:* Guizzardi Giancarlo ; Barry Smith < > ifomis at gmail.com>; Laure Vieu > *Cc:* Nicola Guarino ; João Paulo Almeida < > jpalmeida at ieee.org>; John Bateman ; Stefano Borgo < > stefano.borgo at cnr.it>; Cohn ; Antony Galton < > A.P.Galton at exeter.ac.uk>; Janna Hastings ; > Heinrich Herre ; Werner Kuhn < > werner.kuhn at gmail.com>; Riichiro Mizoguchi ; Mark Musen > ; Leo Obrst ; Barry Smith < > phismith at buffalo.edu>; Zena Wood ; Roberta > Ferrario ; Michael Gruninger < > gruninger at mie.utoronto.ca>; thomas.studer at inf.unibe.ch < > thomas.studer at inf.unibe.ch>; IAOA Executive Council > *Subject:* Re: FOIS and CORE > > > > Dear Giancarlo, All, > > It may be of use to first find out from them why the re-evaluated it > differently. > Good quality papers isn't the only criterion they use. It's also, e.g., > high performers among the authors, but I assume we pass that as well. And > then there's the h-index of the conference, which doesn't do well, as if > FOIS doesn't really have any impact, actually. Well, anyway, that's what it > looked like when EKAW was putting the material together, when Enrico Motta > was showing off his tool for computing citation metrics during the EKAW > steering committee meeting. Anyway, if I had to put in a bet for reason > why, it would be that and a need to spin that story better. Still, to be > sure, perhaps the EC can try to find out why from Lin first? > > Regards, > Maria > > > ---- > > Dr. Maria Keet > > Associate Professor > > Department of Computer Science > > University of Cape Town > > Cape Town, South Africa > > tel: +27 21 650 2667 > > fax: +27 21 650 3551 > > email: mkeet at cs.uct.ac.za > > work: http://www.cs.uct.ac.za > > home: http://www.meteck.org > > > > On 13/04/2021 21:44, Guizzardi Giancarlo wrote: > > Dear all, > > > > Let me raise another concern now. > > Please see the message below from the CORE committee, > > in which they are revising the classification of many CS conferences. > > > As many of you know, CORE is important for computer science faculties. > > Not only directly but also because they build up into several national > evaluation systems. > > If the classification of a conference goes down, people will prefer to > send their > > best papers somewhere else. > > > > According to this revised classification, FOIS went from an A conference > to a B conference. > > I think there is room there for protesting this classification before > their final decisions > > but we need to move fast and institutionally, i.e., IAOA should do it. > > > > Best regards, > > Giancarlo > > > > CORE conference rankings (preliminary) changes > > > Lin Padgham : Apr 11 11:06AM +1000 > > Dear CS colleagues, > The CORE committees have now finished reviewing the approximately 400 new > and existing conferences that were part of this review round. > More than 50% of the conferences reviewed retained their existing rank. > Consistent with our previous practice, we publish here > < > https://docs.google.com/spreadsheets/d/17cnG1Vgyjdu3pGdIHrvyw6HCithPAvTeDd_YB_FVOK0/edit?usp=sharing > > > a list of all planned ranking changes, allowing a short period for any > additional information if community members consider there has been an > error of judgement.* If you wish to potentially question a ranking change, > please notify your intent with an email to lin.padgham at gmail.com > , with subject "Rankings " by April > 18th.* > You will then be provided with the submission and the detailed decision > report, referencing the data on which the decision was based. *If you > consider you have additional data which may change the committee's > recommendation, this must be provided by April 25th*. Committees will then > review this, prior to finalisation and upload of the new CORE conference > ranking. > Regards, > Lin (CORE Rankings Co-ordinator) > > Lin Padgham > Professor in Artificial Intelligence > Computer Science, RMIT University > Melbourne, Australia. > ph. +61 3 9925 3214 (but email works better) > (please note I only work Wednesdays) > > > > > > > > > > _________________________________________________________________________ > Msg Archives: https://listserv.ovgu.de/pipermail/iaoa-advisor/ > IAOA wiki: http://ontolog-02.cim3.net/wiki/IAOA > IAOA website: http://iaoa.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sectionC(1).pdf Type: application/pdf Size: 694489 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sectionD.pdf Type: application/pdf Size: 387371 bytes Desc: not available URL: From gguizzardi at gmail.com Wed Apr 21 07:19:02 2021 From: gguizzardi at gmail.com (Giancarlo Guizzardi) Date: Wed, 21 Apr 2021 07:19:02 +0200 Subject: [iaoa-advisor] FOIS and CORE In-Reply-To: References: <8713385f-0b01-7af7-f428-09dfa93d21a7@irit.fr> <7990605e-b8fd-0fa8-2eb5-641757be7bfb@cs.uct.ac.za> <01ac38d7-3896-7e6b-d197-af7d05c32c48@cs.uct.ac.za> Message-ID: Dear Laure, Some additional information below (just received) best, Giancarlo Providing additional info for CORE rankings Lin Padgham : Apr 16 04:19PM +1000 Dear All, We have now organised a web interface to submit additional information for conferences whose ranks are proposed for change, in particular for conferences where decisions were made based on the limited comparator data. *The url is at portal.core/edu.au/conf-changes . All extra information must be submitted via this process.* These pages have been adapted from parts of the original full submission forms, but with everything made optional. Formatting and wording may be slightly odd as a result. This provides guidance on the data that was requested from full submissions. Any additional data/information can also be provided in the final section, either by typing in directly, or uploading a file. Note that as everything (except conference name and submitter) is now optional, sections B and C will show as completed (green) without any data being added. You are free to answer whichever questions you wish. The Google Scholar section can be safely ignored other than provided the GS h5 if it was not in the original submission. The most useful parts to provide are B2.1, the start of B3, B5 and C. Once finished you must hit submit. You will not receive any confirmation email, but the front page button turning green indicates successful submission. If after submitting you open for any reason, you will need to resubmit, even if no changes have been made. Opening automatically unsubmits. In order to help in understanding the somewhat cryptic data report you will have already received, it is helpful to look at the input form. I provide also here some of the explanation that was also given to committees regarding the PC profile figure in reports and the WPP report and excerpts. The section (Senior) Program Committee in the reports is a profile of the PC (or SPC if a large conference with both PC and SPC). This was obtained by having submitters paste in the PC list from the web, and provide some structured information about the format. This was then processed by our system to obtain the Elsevier h-index of individuals, providing the bar chart profile in the left hand figure. We then extracted the more senior researchers (above a certain threshold) and requested that this list be fed into our WPP tool to see where these more established researchers on the PC were publishing. The right hand figure shows 2 whisker plots: the LHS is the distribution of the h-index of all PC members, showing amongst other things the median h-index. The RHS whisker plot is a distribution of those who were above the threshold - the more established researchers. When the list is provided to WPP that tool (based on dblp data) shows where the individuals in the list are publishing. The data looked at is generally where else the established researchers on the PC are publishing, and to what extent they are publishing in the conference they are on the PC of. The text of the full WPP report is available at the link, while the relevant graphs and a text excerpt are pasted directly into the report. Note that the WPP tool has been modified and now has options to provide journals, conferences, or combined information. Committees also had access (in some cases) to data which we have permission from Elsevier to use by the committees in ranking, but not necessarily to publicize in its entirety. This data gives an analysis of citations and of h-index of most senior author on papers, within the context of the specific FoR code. Methodology here was to obtain for every conference in the CORE DB (where possible) the data for each paper in 2017. Conferences were then partitioned into their FoR code areas and citation centiles were calculated within each area for the top 1%, 5%,10%, 25% and 50%. These were also calculated for the subset of A/A* conferences within a FoR code. By looking at the % of papers from a given conference, compared to these centiles (in particular the A/A* ones), we could then establish how well an individual conference fit the average A/A* profile for that FoR code. We used a similar approach to look at the relative strength of those publishing in a given conference, under the assumption that if many papers are by strong researchers, it is likely to be a strong venue. For this calculation each paper was assigned the h-index of its strongest author, and then centiles were calculated as above. In reports the data most often referenced is the % of papers in a venue at the 25th A/A* centile. If this is much lower than 25%, it is attracting fewer authors with a strong h-index than the average A/A* conference in that FoR code. This analysis was not available for some conferences at the point that reviews were processed. As with all data, this is one piece that must be considered along with other relevant aspects. Committees attempt to form a holistic view based on all available information, both objective data and subjective experience/knowledge. We welcome additional data in situations where community members who have not made a submission on a given conference, consider that they have additional useful information to add, for consideration by the committee. This is particularly important in cases where reviews have been conducted based on more limited comparator data provided by submitters, on conferences they consider less strong than the submission conference. It is important to remember that, to its best knowledge, CORE only includes conferences that are legitimate academic venues with peer review of full papers prior to acceptance. Consequently all included venues (including C and unranked) are considered sound academic events. B conferences are "good to very good" venues, well regarded in their area, while A is a much smaller number of venues regarded as excellent. A* is reserved for flagship conferences in a given area. In general only the most well known top conference of an area will be considered a flagship. If there are multiple top conferences in the same area, only the best will be considered a flagship, unless they are all at about the same level. Not all sub-areas within a FOR code, will have A* conferences, and it may well be that an A conference has equally high quality papers as an A*, but has somewhat less prestige, impact or reach than an A* covering a similar area. Naturally borders are not well defined and there will always be some disagreement around the edges. Committees attempt to be as fair and objective as possible, whilst also avoiding everything percolating to the top. *We welcome additional information on proposed changes, due by midnight 25th April AoE.* *Please make submissions only via the provided interface at http://portal.core.edu.au/conf-changes .* Regards, Lin (CORE rankings coordinator) Lin Padgham Professor in Artificial Intelligence Computer Science, RMIT University Melbourne, Australia. On Sun, Apr 18, 2021 at 11:55 AM Giancarlo Guizzardi wrote: > Dear Laure, > > Sorry for the delay. > Please find it in the attachment > > best, > G > > On Thu, Apr 15, 2021 at 7:45 PM Laure Vieu wrote: > >> Dear Giancarlo, >> >> >> Thanks for this. >> >> If you've got the attachment, could you please forward it? >> (I do not have access to the Google group linked) >> >> >> Best, >> >> Laure >> >> >> Le 15/04/2021 à 09:04, Guizzardi Giancarlo a écrit : >> >> Dear all, >> >> Some additional information on CORE >> >> best, >> Giancarlo >> >> Poviding additional info for CORE rankings >> >> Lin Padgham : Apr 14 07:52PM +1000 >> >> Dear All, >> For those wishing to supply additional information for review of the >> currently recommended changes to conferences in the CORE ranking, I attach >> pdfs of 2 sections of the full submission that was requested for new >> conferences, those wishing to be considered for upgrade, or those listed >> for review. These indicate some information which will be useful to the >> committee, particularly for those conferences that were evaluated on the >> basis of a comparator report only. >> >> I am currently investigating whether we can provide a web interface for >> submission of information (and if we are able to do that, this will be >> compulsory as it greatly assists in getting all the information into the >> DB >> and out to committees). I will post by the end of the week regarding how >> information should be provided. In the meantime these attachments provide >> some guidance as to what information will be of value. >> >> I do note that the section D2 where it asks for a list of at least 20* >> top *people >> in the area, and a report run through the WPP tool, is only of value if >> the >> list of people is chosen in a truly objective, repeatable and verifiable >> fashion, and is without implicit or explicit reference to the conference >> being evaluated. If these instructions are not followed, the report is of >> little value in this process. >> >> Similarly, in section C5, the list of top people regularly involved in the >> conference is only of value if some information is provided as to why you >> consider them a top person in the area, as well as their google scholar >> h-index. Just providing their university affiliation is of little value. >> >> Any additional supporting information can also be provided in any format >> desired. However it is important that information is independently >> verifiable data of some sort, and not simply opinions. While no single >> type >> of data is appropriate as a means of ranking, and a holistic view must be >> sought, there should always be some objective support for views that >> contradict what has been decided on the basis of data currently reviewed. >> >> If you are not planning to potentially provide further information but are >> just wanting to know why particular decisions were made, can i please ask >> that you wait until this information is uploaded to the DB along with the >> new rankings. At this stage there will be a more complete decision report, >> accounting for any extra information received. Also, it is overwhelming >> for >> me to try and respond to everyone, so I would greatly appreciate patience >> until the final reports are uploaded, except for cases where people feel >> that they may be able to contribute additional useful information. >> >> Regards, >> lin (CORE rankings coordinator) >> >> >> >> Lin Padgham >> Professor in Artificial Intelligence >> Computer Science, RMIT University >> Melbourne, Australia. >> ph. +61 3 9925 3214 (but email works better) >> ------------------------------ >> *From:* Maria Keet >> *Sent:* Wednesday, April 14, 2021 4:00 PM >> *To:* Anthony Cohn ; >> Guizzardi Giancarlo >> ; Barry Smith >> ; Laure Vieu >> *Cc:* Nicola Guarino ; >> João Paulo Almeida ; John >> Bateman ; Stefano Borgo >> ; Antony Galton >> ; Janna Hastings >> ; Heinrich Herre >> >> ; Werner Kuhn >> ; Riichiro Mizoguchi >> ; Mark Musen >> ; Leo Obrst ; >> Barry Smith ; Zena Wood >> ; Roberta Ferrario >> ; >> Michael Gruninger ; >> thomas.studer at inf.unibe.ch >> ; IAOA Executive Council >> >> *Subject:* Re: FOIS and CORE >> >> Dear Tony, Giancarlo, All, >> >> On 14/04/2021 15:45, Anthony Cohn wrote: >> >> An interesting discussion and of course I agree it’s worth making a case >> against the “demotion”. I agree about the h-index point – scopus has a >> notion of “field weighted impact factor” – which adjusts for the size of >> the community. I don’t know if scopus computes this for conferences, but >> given the relatively small size of our community this might be an argument >> to make? >> >> agreed, hence the "a need to spin that story better" note in my email. >> afaik, it is something that they consider, but I'm not privy to all those >> details. >> and indeed, it's people beyond the authors' standings and their repeat >> participation, including those of the event organisers and participants, >> among others. >> but as long as we don't know the reasons by it got downgraded, it's >> guesswork why exactly >> >> Regards, >> Maria >> >> >> >> Best wishes Tony >> >> >> >> *From:* Guizzardi Giancarlo >> >> *Sent:* 14 April 2021 14:26 >> *To:* Maria Keet ; Barry Smith >> ; Laure Vieu >> >> *Cc:* Nicola Guarino ; >> João Paulo Almeida ; John >> Bateman ; Stefano Borgo >> ; Anthony Cohn >> ; Antony Galton >> ; Janna Hastings >> ; Heinrich Herre >> >> ; Werner Kuhn >> ; Riichiro Mizoguchi >> ; Mark Musen >> ; Leo Obrst ; >> Barry Smith ; Zena Wood >> ; Roberta Ferrario >> ; >> Michael Gruninger ; >> thomas.studer at inf.unibe.ch; IAOA Executive Council >> >> *Subject:* Re: FOIS and CORE >> >> >> >> Dear Maria and all, >> >> >> >> CORE does periodic revision in its rankings from time to time. >> >> >> >> They have already requested input from the different communities many >> months ago. >> >> As a community, we missed that. I wasn't concerned about that because the >> conference was classified as A >> >> (well, in theory, we could have tried to make the case for A* but that >> would not have worked out as >> >> our demotion shows...). >> >> >> >> The criteria used by CORE is a mixture of objective and subjective points >> (again, the communities >> >> try to make the case considering both types of indicators). FOIS does not >> do well h-index-wise but >> >> that is mainly due to the fact that it is a biannual conference (I won't >> even enter the discussion >> >> of how much of a bad idea is to use h-index to judge conferences...). But >> that is the same for KR >> >> and KR managed to revise their initially bad evaluation. Again, this >> requires an active community effort. >> >> Other conferences that are small but prestigious in their relevant >> communities have >> >> managed to even talk their way up to A* (e.g., PODS). ER is trying the >> same move. >> >> >> >> As for subjective aspects, some of the points that are taken very >> seriously is the impact >> >> and scientific stature of people playing key roles in the conference (PC >> chairs and members, >> >> general chairs, keynote speakers, frequent authors, etc.). >> >> >> >> I think we should react now by contacting them and defending the >> importance of the event >> >> as the most important event for this community and highlighting some of >> these points >> >> >> >> best, >> >> Giancarlo >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ------------------------------ >> >> *From:* Maria Keet >> *Sent:* Wednesday, April 14, 2021 10:33 AM >> *To:* Guizzardi Giancarlo ; Barry Smith < >> ifomis at gmail.com>; Laure Vieu >> *Cc:* Nicola Guarino ; João Paulo Almeida < >> jpalmeida at ieee.org>; John Bateman ; Stefano Borgo >> ; Cohn ; Antony Galton < >> A.P.Galton at exeter.ac.uk>; Janna Hastings ; >> Heinrich Herre ; Werner Kuhn < >> werner.kuhn at gmail.com>; Riichiro Mizoguchi ; Mark >> Musen ; Leo Obrst ; Barry Smith < >> phismith at buffalo.edu>; Zena Wood ; Roberta >> Ferrario ; Michael Gruninger < >> gruninger at mie.utoronto.ca>; thomas.studer at inf.unibe.ch < >> thomas.studer at inf.unibe.ch>; IAOA Executive Council > > >> *Subject:* Re: FOIS and CORE >> >> >> >> Dear Giancarlo, All, >> >> It may be of use to first find out from them why the re-evaluated it >> differently. >> Good quality papers isn't the only criterion they use. It's also, e.g., >> high performers among the authors, but I assume we pass that as well. And >> then there's the h-index of the conference, which doesn't do well, as if >> FOIS doesn't really have any impact, actually. Well, anyway, that's what it >> looked like when EKAW was putting the material together, when Enrico Motta >> was showing off his tool for computing citation metrics during the EKAW >> steering committee meeting. Anyway, if I had to put in a bet for reason >> why, it would be that and a need to spin that story better. Still, to be >> sure, perhaps the EC can try to find out why from Lin first? >> >> Regards, >> Maria >> >> >> ---- >> >> Dr. Maria Keet >> >> Associate Professor >> >> Department of Computer Science >> >> University of Cape Town >> >> Cape Town, South Africa >> >> tel: +27 21 650 2667 >> >> fax: +27 21 650 3551 >> >> email: mkeet at cs.uct.ac.za >> >> work: http://www.cs.uct.ac.za >> >> home: http://www.meteck.org >> >> >> >> On 13/04/2021 21:44, Guizzardi Giancarlo wrote: >> >> Dear all, >> >> >> >> Let me raise another concern now. >> >> Please see the message below from the CORE committee, >> >> in which they are revising the classification of many CS conferences. >> >> >> As many of you know, CORE is important for computer science faculties. >> >> Not only directly but also because they build up into several national >> evaluation systems. >> >> If the classification of a conference goes down, people will prefer to >> send their >> >> best papers somewhere else. >> >> >> >> According to this revised classification, FOIS went from an A conference >> to a B conference. >> >> I think there is room there for protesting this classification before >> their final decisions >> >> but we need to move fast and institutionally, i.e., IAOA should do it. >> >> >> >> Best regards, >> >> Giancarlo >> >> >> >> CORE conference rankings (preliminary) changes >> >> >> Lin Padgham : Apr 11 11:06AM +1000 >> >> Dear CS colleagues, >> The CORE committees have now finished reviewing the approximately 400 new >> and existing conferences that were part of this review round. >> More than 50% of the conferences reviewed retained their existing rank. >> Consistent with our previous practice, we publish here >> < >> https://docs.google.com/spreadsheets/d/17cnG1Vgyjdu3pGdIHrvyw6HCithPAvTeDd_YB_FVOK0/edit?usp=sharing >> > >> a list of all planned ranking changes, allowing a short period for any >> additional information if community members consider there has been an >> error of judgement.* If you wish to potentially question a ranking change, >> please notify your intent with an email to lin.padgham at gmail.com >> , with subject "Rankings " by April >> 18th.* >> You will then be provided with the submission and the detailed decision >> report, referencing the data on which the decision was based. *If you >> consider you have additional data which may change the committee's >> recommendation, this must be provided by April 25th*. Committees will then >> review this, prior to finalisation and upload of the new CORE conference >> ranking. >> Regards, >> Lin (CORE Rankings Co-ordinator) >> >> Lin Padgham >> Professor in Artificial Intelligence >> Computer Science, RMIT University >> Melbourne, Australia. >> ph. +61 3 9925 3214 (but email works better) >> (please note I only work Wednesdays) >> >> >> >> >> >> >> >> >> >> _________________________________________________________________________ >> Msg Archives: https://listserv.ovgu.de/pipermail/iaoa-advisor/ >> IAOA wiki: http://ontolog-02.cim3.net/wiki/IAOA >> IAOA website: http://iaoa.org >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: