[iaoa-swao] My papers again

[Andrea Westerinen]

Latest papers and replies to comments attached.


Andrea Westerinen
T: 425.891.8407
arwesterinen at gmail.com or andreaw at ninepts.com
organizingknowledge.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.ovgu.de/pipermail/iaoa-swao/attachments/20170605/e1ee1d17/attachment-0001.html>
-------------- next part --------------
- - - — -
KenB:

While I liked the paper very much, I have some suggestions for improving the readability of the article.

The article should be more consistent in its use of prefixes. I mention some examples here, but I they are by no means complete.

<arw>Went through and updated all the class and property names with either a prefix, or the use of Schema's, GoodRelations' or ROC's possessive adjective.</arw>

For example, the first paragraph of 4.1 on page 6 is unclear because the classes do not have prefixes. ClimbingGear should be roc:ClimbingGear, Clothing should be gr:Clothing (I assume), etc. Later on page 6, Carabiner should be roc:Carabiner, and the awkward phrase "Good Relations' QualitativeValue" should be simply "gr:QualitativeValue".

Here are some more examples of this. On page 7, "...since the hasCondition property of ROC appears..." should be "...since roc:hasCondition appears..."

<arw>Addressed above.</arw>

Furthermore, the discussion of Condition on page 7 suggests that the roc:hasCondition should be a subproperty of gr:Condition. Is there a reason why this wasn't specified?

<arw>Because gr:Condition is a datatype property and roc:hasCondition is an object property. We would have had to go down the path of literal reification - which still is not = gr:Condition, and that got a bit too complex to be worthwhile (at least in this paper).</arw>

On page 8, "equivalencies" should be "equivalences".

<arw>Changed.</arw>

The paragraph starting "It is assumed..." on page 13 is confusing to read. A little wordsmithing could easily make it clearer.

<arw>Changed to "By default, it is assumed that there is one instance of a product included in an offering. If this is not a valid assumption, then Offering can define the exact quantities using the gr:includesObject property (one of the semantics mapped in Schema). gr:includesObject references an instance of the class, gr:TypeAndQuantityNode. gr:TypeAndQuantityNode exemplifies the use of reification in ontologies (to support n-ary relationships), and has properties, gr:typeOfGood (which references a product or set of products in an inventory), and gr:amountOfThisGood to specify the quantity in the Offer."</arw> 

On page 13, it would be better to say that gr:PriceSpecification is the union of three classes than that it is abstract. The abstract class concept is problematic in an ontology where one can easily construct additional classes (using OWL class constructors, for example) which would have the (usually unintended) consequence of changing the meaning of an abstract class by making it the union of a different set of classes.

<arw>"Abstract" was the word used by GR, but that text has been changed to simply "superclass".  I changed the words similarly.</arw>

On page 16, the phrase "the hasCondition property defined in the ROC climbing ontology" could more succinctly be written "roc:hasCondition".

<arw>Changed</arw>

Footnote 10 on page 17 mentions closed world assumptions used for reasoning. This should be explained a little better, and not confined to a footnote, as it has significance for ensuring that reasoning processes are valid. One way to do this is with the notion of a situation. Situations are a mechanism for supporting closed world semantics within an overall open world without invalidating either one.

<arw>Not changed. As this functionality is provided by Stardog, I cannot comment on how it was accomplished. That is why I provided the reference to the web page where it is described.  I am not comfortable saying more than this and could remove the discussion altogether, but that leaves open questions on how integrity constraints are validated.</arw>

Regarding footnote 11 on page 18, see K. Baclawski, C. Matheus, M. Kokar and J. Letkowski. Toward a Symptom Ontology for Semantic Web Applications. In ISWC'04, pages 650-667. Lecture Notes in Computer Science 3298:650-667. Springer-Verlag. (2004) for possibly the earliest example of this kind of error recording. An interesting aspect of such error recordings is that a single error could cause multiple reports. Abductive reasoning can be used to suggest a common cause for multiple reports. This is just for your information. I am not suggesting that this be included in the bibliography as it is dealing with a tangential topic.

<arw>Thanks. I appreciate the reference!</arw>

On page 20, "a percentage discount that can is set by the retailer" should be either "a percentage discount that can be set by the retailer" or "a percentage discount that is set by the retailer".

<arw>Changed to "can be set".</arw>

- - - - -
MikeB:

Very impressive application that extends real, available ontologies. Some re-work is required on the use of language which is too casual in places. Some intended usages of the ontology are introduced late in the paper – please consider minor restructuring to bring these considerations forward to an initial discussion of scope and requirements and how these informed ontology considerations such as ontological commitment. For example a brief sentence on scoping / use case in the final paragraph of the Introduction section.

<arw>Updated the introduction to include the sentence, 'The goals of the work are to show how ontologies can be integrated with Schema, and to illustrate the value of the integration by describing several, concrete, retail solutions.'  This sentence really describes the scope. I tried various approaches to put more specific scope/use case information into the Introduction - but could not do so without providing all the background that leads up to Section 4. 

In the intro to Section 4, I changed the first paragraph to read, 'The Retail Ontology for Climbing (ROC) was originally proposed as a way to test our methods for knowledge engineering and ontology reuse and integration. However, once initial development was complete, we worked with several retailers to expand it to address better managing their inventory and pricing strategies, and to assist customers regarding their gear and purchases. These use cases are discussed in more detail in the following sections.'</arw>

An example is the decision of whether to segregate things as products from things in themselves – the implications of either choice are apparent i.e. ability to re-use an ontology that does not have such a distinction, and possibly performance issues on one side, versus having instances that selectively use or don’t use particular properties depending on whether they are in retail inventory or personal inventory. As it reads, the reader may conclude you did not consider these issues and the balance of pros and cons but 
simply stumbled upon the implications of this semantic choice.

<arw>This "decision" was actually made by GoodRelations and Schema, and not in ROC or the ODPs. We are reusing their semantics. As this paper is not about critiquing or re-designing GoodRelations - but about using it - I don't feel that I should go into detail about the design decisions of the "standard".</arw>

The introduction to the notion of an ontology in the context of GoodRelations still needs to be stronger. As written, a typical OWL user will see no issue but will not be challenged about the perceived equivalence of “ontology” and “OWL file” whereas the non-OWL using “Applied Ontology” audience will not understand where and why OWL was selected. This issue aims to find common ground between those two kinds of reader. Conflation of “Ontology” with “Ontology in OWL” will get us into trouble with the AO editors.

<arw>Section 3 includes a paragraph that explains what an ontology is, what an ODP is and why OWL 2 was chosen for ROC. I am not sure what more I can say since I cannot make any statements about why RDF/XML (i.e., OWL) was chosen for GoodRelations. If you could make some specific suggestions for changes to the paragraph, that would be great. Here it is: 'Whereas an ontology describes the concepts, relationships, properties, axioms and individuals of a domain, an ODP is that same information but focused on a specific and commonly recurring modeling problem. In this paper, both the semantics of a set of retail ODPs and their rendering using OWL 2 ("OWL 2", 2012) are discussed. For the retail ODPs, OWL 2 was chosen since it is built on RDF, which is an underpinning of the Linked Data and schema.org environments.'

Language:
The word "underwhelming" is only intended to be used as ironic construction. Please do not use it in this paper, as it sets a casual tone that would be out of keeping with the Special Edition. Please keep all language formal.

<arw>Removed.</arw>

Schema – as an abbreviated name for schema.org – please introduce this usage the first time it is used

<arw>This was stated in the second sentence, second paragraph of the Introduction ('The data is structured by the semantics of the schema.org vocabulary (hereafter referred to as "Schema")').</arw>

References in text – e.g. OWL 2 ("OWL" 2012)
should use a numbered reference to the entry in the References section, viz:
OWL 2 (9)

<arw>But this is not what is required by APA guidelines - specifically, 'in-text citations consist of the surname(s) of the author(s) and the year of publication - if there is no author, use the title (or a short form of the title, if it is lengthy) and the year'.</arw>

Page 6:
ROC will be available in full on GitHub in 2017.
Not sure this is relevant, at least without a reference or link.

<arw>Removed</arw>

Introduction
GoodRelations is introduced simply as “an ontology”. Please briefly explain what sort of ontology it is e.g. this is an ontology that uses OWL, or OWL-DL or RDF Schema or whatever. That is, what does it mean to be an ontology at the point where this word is introduced with reference to GoodRelations. Do this before we get to the mention of ontology design patterns so this has a context.

<arw>I added text clarifying that GR is an RDF/XML (OWL) ontology. I don't know what more to say, as I noted above in reply to an earlier comment.</arw>

Also some discussion needed on intended scope and the implication of this on design and re-use – see comments on 4.5 below.

<arw>Replies below.</arw>

2. GoodRelations Ontology in schema.org
You note that “the semantics behind the words remain roughly the same”. Elsewhere in schema.org words are often used polysemically. Is that the case for the words used (or replaced) from GoodRelations, given that the latter is an ontology and therefore not polysemic?
If this is not known, consider changing this to read that the concepts (not the words) remain roughly the same. This fits better with the paragraph and listings that follow.

<arw>Changed to the "semantics of the concepts".</arw>

3. Retail Ontology Design Pattern
Semantics:
Is there any attempt, either in GoodRelations or in the Climbing ontology, to distinguish between a thing in itself and a thing as offered as a product? 

<arw>Yes, this is discussed in the distinctions between Section 4.3 (Product Model Ontology) and Section 4.4 (Instances in a Retailers Inventory).</arw>

If not, is it explicitly intended that this ontology not be reusable outside of the context of retail? 

<arw>As noted in the overall Introduction, GoodRelations is defined for businesses and e-commerce. As noted in the introduction to Section 4, ROC is targeted at climbing gear retailers. Although concepts could be reused, I clarified Section 4.5 (personal inventory -> personal inventory of purchased items) to be more about the retailer maintaining a list of the purchased items.</arw>

Was this choice considered e.g. balance of efficiency in conflating things with things-as-products versus reusability in non retail contexts (e.g. safety / incident reports)? 

<arw>Even in the case of safety/incident reports, the retailer would use the personal inventory to notify the customer. This still comes down to a retail use.</arw>

Was there a conscious decision to retain the GoodRelations conflation of these concepts in order to support re-use?

<arw>Actually, GR does not conflate the concepts, but clearly distinguishes a product from an instance of the product - as noted in Sections 4.3 and 4.4.</arw>

See also my comments on 4.5 where you do in fact introduce a second, non retail usage.

4.1 Climbing Domain Ontology
Really good explanation about the use of enumerations in ROC as opposed to literals in GR. It is worth also highlighting that this makes the ROC ontology one about the things in the domain rather than data literals about the things in the domain – highlighting a difference in model theory between GR and ROC and showing how this influences what you did. Not critical, just a suggestion.

<arw>I did not add more text (sorry) since the paper is already pretty long and targeted at a Schema.org user. I would rather not add too much more theory.</arw>

4.5 Personal collection catalog
It is unclear why this might exist – as you note later in this sub section. Better to start with the rationale.

<arw>We have not had anyone use the personal inventory - although the retailers did find a use.  I updated Section 4.5 to discuss this.</arw>

Also this is where the semantics of the thing in itself versus thing-as-product becomes apparent. You mention that one property in the ROC ontology, hasCondition, is only used for personal collections but could be used for resold products.

<arw>Clarified the use of hasCondition . . . 'Beyond the date of purchase, the condition of the gear could be captured using the roc:hasCondition property. This is important if the retailer resells used equipment. However, the retailer could also allow a user to maintain the condition of the equipment in their personal inventory. A motivation for labeling products in poor or damaged condition would be to receive notification when there are sales on replacement gear.'</arw>  

If there is a motivation to use the ontology for personal collections as well as retail, this should be part of the up-front discussion linking intended uses of the ontology with deciding on these kinds of ontological commitments in light of those use cases. As a practical result for instance, do you have a product class with the properties related to the sale of that thing being optional or do you now have a different class for the thing in the collection, without inventory properties? Doesn’t this loosen the semantics on the retail side? Again the rationale is needed.

<arw>As noted above and shown in Figure 3, there are different superclasses for ProductModel and Individual, with different properties. This is as designed in GoodRelations.</arw>

I see that creating purchase lists is an important use cases – again, to the up front scoping and design discussions – this kind of thinking doesn’t emerge during the design process but is the set of business requirements you started with. The paper should try to reflect this progression of thought.

<arw>This is highlighted in the intro to Section 4. '. . . we worked with several retailers to expand it [the ontology] to address better managing their inventory and pricing strategies, and to assist customers regarding their gear and purchases. These use cases are discussed in more detail . . .'</arw>

- - - - -
ToddS:

0) I still think there's an issue with the way references are made.
In some cases parenthetical references are given in others
endnotes are used. Have you consulted the IOS reference for
attributions and references (which in turn points somewhere
else)?
<arw>Yes, I reviewed the requirements (APA style guidelines). Web sites can be referenced 'in-line' ("For a passing reference to a website in text, the URL is sufficient; no reference list entry is needed.").</arw>

1) In the first sentence of the second paragraph of section 1,
"Search results are improved by embedding "structured data" markup within a web site's
HTML." 'HTML' should be changes to either 'HTML content', or better yet
'web site's pages'.

<arw>Changed to 'pages'.</arw>

2) In the second sentence of the third paragraph of section 1,
"... over a third of all Google search result ...", I think 'result' should
be plural, 'results'.

<arw>Changed.</arw>

3) In the first sentence of the sixth paragraph of section 1, the term
'overviewing' should be replaced.

In the same sentence there's an inconsistency with an assertion in
a preceding paragraph, " ... then defining Ontology Design Patterns and ...".
In paragraph 4 of section 1 there's the following assertion,
"... this paper defines a simple Ontology Design Pattern".
That is only a single design pattern in defined in the paper.

The construction 'proceeds by' is also unfortunate. Both my concern and yours are, I think, reflections of the implicit and discomfiting anthropomorphizing of the word paper. Papers are inanimate.  An author might proceed, but paper's aren't known for their mobility. I'd probably strike the notion of procedure altogether.

But to your point: the verb "overview", from Wiktionary means: "To engage in an overview; to provide a brief summary". Again, an inanimate or passive object like a  paper might contain an (noun) overview but wouldn't  be able to (verb) overview anything.

So here is my suggestion (brackets show alternatives that may be needed depending upon context):

 "[In their paper], the author [or authors] then [proceed to] provide[s] an overview of ....", or shortest form "The author then provides an overview of ..."

<arw>Revised to read: 'The paper is structured as follows: Section 2 is an overview of the GoodRelations components of Schema, while Section 3 is an overview of Ontology Design Patterns in general and the authors' proposed retail patterns. Section 4 . . .'</arw>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IntegratingGRInADomainOntology-FinalRevision.pdf
Type: application/pdf
Size: 1539640 bytes
Desc: not available
URL: <http://listserv.ovgu.de/pipermail/iaoa-swao/attachments/20170605/e1ee1d17/attachment-0002.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: WorkingOnOntologies-RevisedApril.pdf
Type: application/pdf
Size: 1605732 bytes
Desc: not available
URL: <http://listserv.ovgu.de/pipermail/iaoa-swao/attachments/20170605/e1ee1d17/attachment-0003.pdf>
-------------- next part --------------
- - - - -
KenB:

My review was brief.  Here it is:

The word "meryonymic" should be "meronymic"

<arw>Changed.</arw>

On page 4, near the bottom, "More just manipulating" should be "More than just manipulating"

<arw>Changed.</arw>

The survey of different tools for ontology development is the best part of the paper.  The subsequent part is work in progress that shows what the issues are and discusses possible solutions but does not completely solve any of the problems.  I grant that this is interesting, but it is rather too speculative for an archival journal.

<arw>As discussed at the last meeting, changed the text to be less about what we could do versus what we did and why.</arw>

It would be helpful to include a few sentences in the introduction that explain how this paper is relevant to the special issue.

<arw>I added the following paragraph to the Introduction, 'The role and engineering of ontologies for the Big Data and Linked Data communities were two of the basic problems addressed in the Ontology Summit 2014 Communiqué (Gruninger, Obrst et al., 2014). However, in order to use ontologies, they must be understandable and accessible to the members of the communities, and correctly reflect the necessary domain concepts. This requires that the concepts and relationships in an ontology be presented in a way that is familiar to the users and the experts.'</arw>

- - - - -
MikeB:

An important paper which sets out a vision for presenting ontology content to domain experts, something that is often ignored or underestimated in the field. The range and type of domain review artifacts is in line with my own experiences and efforts on FIBO. There are numerous grammatical and styling issues, and the recurrent problem of assuming the reader assumes that to be an ontology is to be an OWL ontology – that has to be rectified in order to be in line with the AO editorial policy, which also discourages detailed technical discussions – however the technical content here is important for understanding how the visualization techniques are made to work. The final section sets out directions for future work and the abstract and introduction should mention this. Also be careful about your assertions about UML - this has a very specific meaning and I think what you mean is UML-like.

<arw>Specific changes are discussed below.</arw>

Details below:

Ontology versus OWL Ontology

The introduction seems to conflate the notion of “what is an ontology” with “What is an ontology in OWL” as though OWL is the only applicable ontology language and that anyone who is engaging with ontology is doing so using the language of OWL. This needs to be reframed, starting with a clear statement of what the authors consider to be “an ontology” and describing the Protégé and OWL ecosystem as an example of wisely used syntax and tooling to achieve these ends, not a necessary precondition to doing ontology. See my suggested introduction to Section 2.2, which could even be brought forward to the earlier introduction.

See e.g. Section 2.1 for the right kind of language: “…formal language such as OWL”

<arw>Rewrote some of the sentences in the Introduction, such as 'Even the use of the word, "ontology", conveys complexity and the need to learn new ways of expressing and representing concepts.  Description or common logic languages, modeling methodologies, and ontology development tools, while highly useful, take time to comprehend and require experience to use effectively.'</arw>

Section 2.2 Visualizing Ontologies
Preface this discussion with a statement that

“In this section, only visualization for OWL based ontologies is discussed. Other ontology formats may have their own visualization methods but the ubiquity of the OWL ecosystem for tooling and ontology modelling means that RDF and OWL visualization tools are of the most immediate interest for collaborative work on ontologies”

Or something like that.

<arw>I added an opening paragraph to Section 2.2 . . . 'There are various techniques to define ontologies in an unambiguous, computer-understandable way. Here, we focus on the use of the Resource Description Framework ("RDF", 2014) and Web Ontology Language ("OWL", 2012) due to the ubiquity of the OWL ecosystem for tooling and development. This means that RDF and OWL visualization tools are of the most immediate interest for collaborative work on ontologies.'</arw>

Language and Styling
Typos / Grammar
"The steps in the UPON Lite methodology are discussed in more detail in Section 4. And,
it is interesting to note that its supporting tooling is a spreadsheet, which is discussed in Section 2.3."

Lose the “And, “ construction, also the “which” is a bit vague. Try:

"The steps in the UPON Lite methodology are discussed in more detail in Section 4. It is interesting to note that its supporting tooling is a spreadsheet, the use of which is discussed in Section 2.3."

<arw>I removed the second sentence in response to BrandonW's comments below.</arw>

References
Should be moved from footnotes and added to the References section.

<arw>The guidelines state that 'For a passing reference to a website in text, the URL is sufficient; no reference list entry is needed.' So, I moved most of the references to the back section (since they are more than passing references) and for Protege, added the URL in-line (since it is only referenced once).</arw>

Style of references should be a number [1] corresponding to the references number in the References section. Where this is not a published paper or book, please give the web address along with the date this was retrieved (1. Blah de Blah, from www.abc.de retrieved on 03 March 2017)

<arw>Actually this is not what the APA Guidelines state. They require . . . 'APA in-text citations should include the author's last name followed by the year of publication. All publications cited in the text should be presented in an alphabetical list of references at the end of the manuscript.'</arw>

Specific Comments
2.3 Ontologies and Spreadsheets
Have you looked at ANZO from Cambridge Semantics? I know you say there are many more but this is one that is getting a lot of traction.

<arw>Yes, but it did not add anything and is not open-source. I did not want to start going down a path of vendor-specific products.</arw>

3. Ontograph

Where it says
"But, a single diagram, such as UML, can also generated."

Do you mean it generates a graph that is explicitly in UML (e.g. UML Class Diagram notation), or do you mean generating single diagrams in the same way UML does, but for OWL. I assume the latter but it reads like the former. Also a “be” is missing. Also don’t start a sentence with But. Try

"A single diagram, as with UML, can also be generated."

<arw>Changed the wording to 'It is also possible to generate a single, UML-style class diagram.'</arw>

P7:
"Note that the "Graph Type" selection, shown in the figures above, defines the content to
be graphed (class/inheritance information, or a property, instance/enumeration or UML
diagram). It is possible to select more than one type (in which case, all the generated . . ."

UML is a very specific modeling language and its semantics are not the semantics of OWL. There are deep an unresolved conversations within the OMG itself about whether e.g. OWL classes may be considered as extensions of UML classes or not. You will be opening this up to a whole world of pain if you simply say “UML Diagrams” unless they are somehow compliant to the UML metamodel. And if they are, they are likely not good ontology models. There are currently 2 ways of extending or aliasing UML to provide visual representation of OWL model content: the Ontology Definition Metamodel (ODM) and the Semantics for Information Modeling and Federation (SMIF) models. Any discussion of generating “UML diagrams” would have to relate to one or other of these or be a third alternative to what they have done – a very ambitious claim.

If you say “UML-style” diagrams throughout, this should insulate you from that.

Unless you provide XMI output that can be ingested into a UML tool – effectively transforming the OWL ontology into a UML data model?

<arw>Thanks for catching this.  I changed everything to "UML-style".</arw>

Section 4 Conclusions and Future Work
I think this is good, but since this is setting out possible future thinking, I would add something in the abstract and the introduction to the effect that this paper is also intended to stimulate discussion on future directions for the techniques described here. I think this is important work.

<arw>I added a sentence to the abstract, and concluded the introduction with this paragraph . . . 'The following paper reviews development efforts in this space. Related work is reviewed in Section 2. We describe our work and experiences with a custom graphing tool (OntoGraph) and supporting textual documentation in Section 3. Then, in Section 4, we discuss how the OntoGraph and spreadsheet tooling has evolved, and areas of investigation for future development. One of the goals in presenting this work is to stimulate discussion on the requirements, technologies and techniques involved in working on ontologies with domain experts.'</arw>

- - - - -
BrandonW:

Thank you for submitting the paper, "Ontology Development by Domain Experts (Without Using the "O" Word)". This paper suggests that giving domain experts the ability to create ontologies, which otherwise would have been created by ontology experts (ontology engineers), is a useful endeavour. The authors illustrate this through examples of tools developed by the authors (or within their LLC), specifically OntoGraph and a worksheet application which lays out classes, properties, relationships, etc. in a spreadsheet format for users (domain experts) to fill in. Of particular interest is using spreadsheets for ontology authoring and a web based graph generator for visualizing ontologies, and/or specific components (classes, properties, etc.) of an ontology. These tools were created and devised to reduce the gap between ontology authoring and editing tools and commonly used, or ubiquitous, tools like spreadsheets.

Overall, this paper raises some interesting questions as to what it means to create an ontology---and, of course, by extension, what an ontology actually is (the elusiveness continues). Learning how to create an ontology does have a fairly steep learning curve. In fact, a previous ontology summit was dedicated to delineating the topics a student would have to be exposed to in order to be considered knowledgeable in all the relavent topics associated with ontology development---it was not a small list.

I did find parts of this paper to be potentially contradictory. In many respects ontology development is simply an interdisciplinary exercise between someone who understands the logic and modeling methodologies (and likely RDF/OWL) and someone else focused and/or trained in a different domain. This appears evident in the paper as the domain experts described are not actually creating the ontology, they simply populating it with values. There is a distinction. The domain experts are not making new relationships and developing the semantics of those newly created entities.

<arw>Having domain experts create new classes and properties is a future goal of the work, as noted in Section 4. 'The OntoGraph and spreadsheet tooling assumes that an appropriate, domain ontology exists for a domain, and is defined using an OWL syntax. We are extending the tooling to capture new concepts and properties (and their related data) from spreadsheets, as well as comments and tracked changes on existing concepts and properties. (Also important in this approach is to capture the provenance of the additions and changes.) The intent is to follow the workflow defined by UPON Lite . . .'</arw>

The authors allude to an advantage to increasing efficiency (and cost) by reducing complexity, but there is much work still needed to address the potential efficacy of this approach. It would be great of the authors addressed this somewhere in the work.

<arw>I am not sure where increased efficiency is mentioned (or alluded to). The only claim is about the need to have ontologies reviewed by domain experts, who are not versed in any of the formal logics or logic languages. If there is a specific text that needs clarification, I would be happy to change it.</arw>

My main concern regarding this submission is, what does this approach add, or do differently and/or better, as compared with other tools in this space---the ones mentioned as well as other spreadsheet (CSV, Excel, etc.) to RDF applications? An expert can set up a spreadsheet with arbitraty column headings that are not necessarily part of the RDF or OWL nomenclature. It's unclear what happens next, and how that is used and assessed (or validated).

<arw>In Conclusions, I clarified the first paragraph to read . . .'Section 3 focused on tooling to document existing OWL ontologies and share their content beyond ontology experts. The tooling is unique from that discussed in Section 2 in that it:

•	Combines both visual and spreadsheet (textual) output, versus being focused only on visualization or only using spreadsheet data
•	Provides visual output that can be customized to business or industry conventions, versus mandating a specific visualization format
•	Can simplify visualizations by collapsing multiple edges between two nodes to a single edge
•	Is general purpose, versus targeted at a particular industry or domain
•	Can create flexible spreadsheet outputs, versus restricting spreadsheet cells, columns or rows using templates and hard-coded conventions
•	Supports the current version of OWL (OWL 2)
•	Is available and maintained as open-source (https://github.com/NinePts)'</arw>
 
As an aside, it would be interesting to see how this method compares to ML classification or other frameworks like OntoLex and/or Lemon? This is perhaps stretching the essence of the paper, but acknowledgement might be appropriate if there were a brief discussion section.

<arw>The tooling really has nothing to do, at this point, with ML classification, NLP or lexicons/word senses.  I am not sure how to discuss these concepts as they are currently unrelated.</arw>

comments by section:

1. Introduction
-A good, simple introduction alluding to the amount of resources it takes to create an ontology using domain experts along with a knowledge engineer (or several of each).
-In addition to the "grey" literature, is there a link to the actual NASA ontology?

<arw>There are some articles by one of the scientists mentioned in Earley's paper, but I have not found a specific link to the NASA ontology. Also, this was only referenced as an example and the ontology is not used further in the paper.</arw>

-No need to call out Protege on the second page; the complexity issue holds with all ontology authoring tools---Protege, TopBraid, NeOn, etc., as well as other "vocabulary" or CV tools, for example VocBench.

<arw>Removed the reference to Protege.</arw>

-In the third paragraph, the issue with validation is conflated here. The validation is both ways. The domain expert is checking that the model is describing the semantics of the domain, and the ontologist is (likely) ensuring the logic is valid. It's a symbiant relationship illustrating accuracy vs precision.

<arw>Since this paper is focused on review of ontologies by domain experts (and not logic review by ontologists), I purposely did not raise the reverse validation as it would not have added anything.</arw>

2. Related Work
-Clearly states it's an overview.
-The characteristics of success defined by Bada et al., 2004 is interesting
-In the second paragraph, what is the "backing methodology"?
-Similarly, which method did UPON Lite extend?

<arw>Rewrote the second paragraph to read . . . '. . . choice of an ontology development methodology is extremely important and will impact the entire scope of a project. Many methodologies have been proposed, catalogued and extended over the years (Corcho, Fernandez-Lopez, & Gomez-Perez, 2003, and Sure, Tempich & Vrandecic, 2006). One such methodology, UPON Lite (De Nicola & Missikoff, 2016), extends the Unified Process for Ontology building (UPON) and directly relates to the problems described above. UPON is a cyclical ontology-building method that utilizes Unified Process (UP) and the Unified Modeling Language ("UML", n.d.) for iterative, incremental, and use-case driven development (De Nicola, Missikoff, & Navigli, 2005). Even so, UPON is still dependent on ontology engineers and requires extensive knowledge of modeling techniques. UPON Lite was developed to address the "growing need for simpler, easy-to-use methods for ontology building and maintenance, conceived and designed for end users, … reducing the role of (and dependence on) ontology engineers".'</arw>

-It would be great to name some of the development methodologies.

<arw>If the reader is really interested, they could go to the referenced paper. I am not sure what value a list of methodologies (by name) would provide.</arw>

-The last sentence in 2.1 is re-stated in 2.3. It seems to fit better with the content in 2.3.

<arw>Removed the sentence from Section 2.1.</arw>

-The visualization section is informative. I definitely learned something.

-The second paragraph in section 2.3 isn't needed as it is redundant.

<arw>The goal of the paragraph was to indicate that this is definitely NOT an exhaustive list of tools. I feel that it is important to state this.</arw>

-A CSV is not a spreadsheet. This informaton does seem relevant as CSV is a fairly ubiquitous storage/transfer format. Perhaps amending the section heading would help.

<arw>Clarified the paragraph to read . . . 'A simple approach to importing or exporting data to a spreadsheet is as a set of comma-separated values (CSVs).' Personally, I have done my fair share of exporting a spreadsheet to CSV and converting to an ontology.:-)</arw>

-There is a fair bit of text regarding ROBOT that seems to me to be extraneous.

<arw>The text about ROBOT was the concept of a "template". This is key to understanding how ROBOT works, and also what makes it difficult to set up and different from the OntoGraph/Spreadsheet tooling.</arw>

-is it fair to say the ROBOT, RightField and Populous add DB style restrictions to spreadsheet cells? It seems that is what is being stated. Making that explicit would be helpful.

<arw>I would not say that the listed tools add "DB style restrictions". For me, the latter means integrity constraints and specific datatype requirements. But, the tools provide ways to convert CSV and tab-separated data from spreadsheets to OWL. The templates are the conventions that allow this. For example, for ROBOT, the first line/row must contain names for every column used in the data. The second line/row contains template strings for each column that will be used in the OWL conversion. Then, the remaining rows correspond to OWL classes or individuals. This is stated in the paper.</arw>

3. OntoGraph and Spreadsheet Tooling for Domain Experts
-OntoGraph seems like it addresses some common issues with ontology visualization---especially as it relates to published artefacts.
-Did the settings in figure 1 create the image in figure 3, and do figures 2 and 4 have a similar relationship? It would be great if that were explicit---or if there were an example like that for the user to follow.

<arw>Yes. Modified the text to indicate the customization options that were used.</arw>

-figures 3 and 4 have about 10 prefixes present. I think only three are used, and some nodes do not have a prefix (i.e. :SpatialThing)

<arw>Fixed the missing prefixes (they were not originally defined in FOAF). The code lists all prefixes. They are all used in the properties diagram.</arw>

-I would really like figure 4 and 7 to be much bigger---i.e. readable.

<arw>Me too, but it won't fit on a page.</arw>

The OntoGraph and Spreadsheet Tooling functionality should be in their own sections.

<arw>I separated the discussion into 2 sub-sections.</arw>

-The spreadsheet tooling needs unpacking. The domain experts aren't really developing an ontology, they are simply filling in values to a structure that has been previously set up using rdf and owl components. 

<arw>Yes, this is addressed further in Section 4.</arw>

Also, the SPARQL query sets up the headers for the spreadsheet, or is there meant to be class and relationship information in there already for the domain experts to use (as a go by)?

<arw>Section 3 is clarified to state that the query output was manipulated by a specific program, OntoSheet, which is discussed further in Section 4.</arw>

-How is the informaton in the spreadsheet pulled back out of the spreadsheet and into the ontology---CSV to OWL, for example.

<arw>Discussion added to Section 4. 'We are extending the OntoSheet tooling to capture new concepts and properties (and their related data) added to the worksheets, as well as any review comments defined for existing concepts and properties. (Also important in this approach is to capture the provenance of the additions and changes.)'</arw>

-What is the benefit of using this approach to some of the other approaches, either listed in section 2 or other CSV to RDF and/or OWL tools?

<arw>Modified the start of Section 4 to clearly state the benefits and differences.</arw>

-How might a user add new information to the structure (similar to what Populous addressed)?

<arw>Noted in the new text about OntoSheet in Section 4 - as this is still in development.</arw>

4. Conclusion or Future Work
-where are the tools being released as open source projects---github, sourceforge, gitlab, etc.?

<arw>Provided details. Source is on schedule to be available in May.</arw>

-why is figure 7 in section 4? Surely this should be included in the previous section.

<arw>Moved to Section 3.</arw>

In the third paragraph, the first sentence states, "The OntoGraph *and* spreadsheet tooling assumes that an appropriate, domain ontology exists for a domain, and is defined using an OWL syntax". This is key information and should be one of the first things in section 3.

<arw>Modified the discussion of OntoGraph to read . . . 'For visualization, we created the OntoGraph program ("OntoGraph", 2016-2017). It was designed to provide documentation on existing OWL ontologies, developed for our consulting customers.' And modified the spreadsheet sub-section to begin with . . . 'Beyond graphs, we also generate domain documentation for OWL ontologies, based on a simple spreadsheet format.'</arw>

general notes:
citations---I assume these will be numbered and the full author date was for reviewing purposes. If that is not the case, it probably should be.

<arw>As stated above, the submission requires the use of the APA guidelines for citations.  These were followed (for good or for bad).</arw>