Eden.dei.uc.pt

The Paradoxical Role of Similarity in Creative Reasoning
Nuno Seco, Tony Veale and Jer Hayes
Abstract
In this paper we present a semantic similarity metric that wholly relies on the hierarchical structure of WordNet which makes it amenableas a means of evaluating creativity when considering creative recategorizations of concepts in an Ontology (Veale, 2004). Many creativediscoveries are only acknowledged long after their conception due to changes in the evaluation criteria (Bento and Cardoso, 2004),therefore evaluation plays a critical role in creative reasoning systems. We evaluate the similarity function and report a correlation valueof 0.84 between human and machine similarity judgments on the dataset of (Miller and Charles, 1991), which is suggestively close to theupper-bound of 0.88 postulated by (Resnik, 1999). We then use the similarity metric as basis for evaluating some examples of creativecategorizations. An extension of the metric is also suggested as a means of assessing analogical similarity by looking for analogical cuesin the taxonomy.
Introduction
words that denote both animals and the meat derived from Creativity is a vexing phenomenon to pin down for- them (e.g., chicken, lamb, cod), and this polysemy reflects mally (Wiggins, 2003), which is perhaps why we tend the transformation potential of animals to be used as meat.
to think of it in largely metaphoric terms. For example, (Veale, 2004) further points out that if one can iden- creativity is often conceived as a form of mental agility tify all such instances of function-transforming polysemy that allows gifted individuals to make astonishing mental in WordNet, we can generalize from these a collection of leaps from one concept to another (Hutton, 1982). Alter- pathways that allow a system to hypothesize creative uses nately, it is popularly conceived as a form of lateral think- for other concepts that are not so entrenched via polysemy.
ing that allows those who use it to insightfully cut sideways For example, WordNet defines several senses of knife, one through the hierarchical rigidity of conventional categories as an edge tool used for cutting and one as a weapon used (de Bono, 1994). Common to most of these metaphors is for injuring. Each sense describes structurally similar ob- the idea that creativity involves recategorization, the abil- jects (sharp flat objects with handles) with a common be- ity to meaningfully move a concept from one category to havior (cutting) that differ primarily in function (i.e., slic- another in a way that unlocks hidden value, perhaps by re- ing vs. stabbing). This polysemy suggests a generalization vealing a new and useful functional property of the concept.
that captures the functional potential of any other edge tool, For example, psychometric tests such as the Torrance test of such as scissors and shears, to also be used as a weapon.
creative thinking (Torrance, 1990) try to measure this abil- Some recategorizations will exhibit more creativity than ity with tasks that, e.g., ask a subject to list as many unusual others, largely because they represent more of a mental leap and interesting uses of old tin cans as possible.
within the ontology. We can measure this distance using The ad-hoc nature of creativity is such that most ontolo- any of a variety of taxonomic metrics, and thus rank the gies do not and can not provide the kinds of lateral linkages creative outputs of our system. For instance, it is more cre- between concepts to allow this kind of inventive recatego- ative to reuse a coffee can as a percussion instrument than rization. Instead, ontologies tend to concentrate their repre- as a chamberpot, since like tin can the latter is already tax- sentational energies on the hierarchical structures that, from onomized in WordNet as a container. Any similarity met- the lateral thinking perspective, are as much a hindrance as ric (called σ, say) that measures the relative distance to the an inducement to creativity. This is certainly true of Word- Most Specific Common Abstraction (MSCA) will thus at- Net (Miller et al., 1990), whose isa hierarchy is the most
tribute greater similarity to coffee can and chamberpot than richly developed part of its lexical ontology, but it is also to coffee can and tympan. This reckoning suggests that the true of language independent ontologies like Cyc (Lenat creative distance in a recategorization of a concept c1 from and Guha, 1990), which are rich in non-hierarchical rela- α to ϕ may be given by 1 − σ(α, ϕ).
tions but not of the kind that capture deep similarity be- Of course, distance is not the only component of cre- tween superficially different concepts. It is connections like ativity, as any recategorization must also possess some util- these that most readily fuel the recategorization process.
ity to make it worthwhile (e.g., there is a greater distance Withal, (Veale, 2004) has suggested several ways of de- still between tin cans and fish gills, but the former cannot tecting these lateral linkages in WordNet by exploiting ex- be sensibly reused as the latter). In other words, a creative isting polysemies. Polysemy is a form of lexical ambiguity product must be unfamiliar enough to be innovative but fa- in which a word has multiple related meanings. The form miliar enough to be judged relative to what we know al- of polysemy that interests us most from a creativity per- ready works. This is the paradox at the heart of ontological spective is function-transforming polysemy, which reflects creativity: to be creative a recategorization must involve a at the lexical level the way concepts can be extended to ful- significant mental leap in function but not in form, yet typi- fill new purposes. For instance, English has a variety of cally (e.g., in WordNet), both of these qualities are ontolog- ically expressed in the same way, via taxonomic structure.
Information Theoretic Approaches
This suggests that the similarity σ must be simultaneously A recent trend in Natural Language Processing (NLP) maximized (to preserve structural compatibility) and mini- has been to gather statistical data from corpora and to rea- son about some particular task in the light of such data.
Fortunately, polysemy offers a way to resolve this para- Some NLP systems use a hybrid approach where both dox (Veale, 2004). If a creative leap from α to ϕ is fa- statistics and a hand-crafted lexical Knowledge Base, such cilitated by a polysemous link between β and χ where β as WordNet, is used. SS has been no exception to this trend.
is a hyponym of α and χ is a hyponym of ϕ, the sen- Despite this movement, we feel that these knowledge bases sibility of the recategorization of c1 can be measured as have not yet been fully exploited, and that there is still much σ(c1, β) while the creativity of the leap can be measured as reasoning potential to be discovered. Hence, we present a 1 − (α, ϕ). The value of a creative product will be a func- novel metric of IC that is completely derived from WordNet tion of both distance and sensibility, as the former without without the need for external resources from which statis- the latter is unusable, and the latter without the former is tical data is gathered. Experimentation will show that this banal. The harmonic mean is one way of balancing this new metric delivers better results when we substitute our IC values with the corpus derived ones in previously estab-lished formulations of SS.
Previous information theoretic approaches ((Jiang and 1, β) × (1 − σ(α, ϕ)) Conrath, 1998), (Resnik, 1995) and (Lin, 1998)) obtain the 1, β) − σ(α, ϕ) needed IC values by statistically analyzing corpora. They Considering the example of an ax being categorized as associate probabilities to each concept in the taxonomy a weapon would lead to the following instantiation: based on word occurrences in a given corpus. These proba- bilities are cumulative as we go up the taxonomy from spe- cific concepts to more abstract concepts. This means that every occurrence of a noun in the corpus is also counted as an occurrence of each taxonomic class containing it. The IC value is then obtained by considering the negative log • χ = knife (the weapon sense) res(c) = −log p(c) where c is some concept in WordNet and p(c) is its prob- It is precisely the issue of Semantic Similarity (SS) that ability according to its frequency in a corpus. It should this paper will address. We present a wholly intrinsic mea- be noted that this method ensures that IC is monotonically sure of similarity that relies on hierarchical structure alone.
decreasing as we move from the leaves of the taxonomy We report that this measure is consequently easier to calcu- to its roots. (Resnik, 1995) was the first to consider the late, yet when used as the basis of a similarity mechanism use of this formula, that stems from the work of (Shannon, it yields judgments that correlate more closely with human 1948), for the purpose of SS judgments. The basic intuition assessments than other, extrinsic measures that additionally behind the use of the negative likelihood is that the more employ corpus analysis. Given the hierarchical nature of probable a concept is of appearing then the less informa- our metric we argue that it is an ideal candidate for the role tion it conveys, in other words, infrequent words are more of σ presented in equation 1.
informative then frequent ones. Knowing the IC values for This paper is organized in the following manner; in sec- every concept allows us to calculate the SS between two tion 2. we provide a brief overview of some of the ap- given concepts. According to Resnik, SS depends on the proaches that we believe are increasingly relevant to our amount of information two concepts have in common, this research and that base themselves on the notion of Infor- shared information is given by the MSCA that subsumes mation Content (IC) (Resnik, 1995) which is the corner- both concepts. In order to find a quantitive value of shared stone of our metric. These approaches are usually dubbed information we must first discover the MSCA, if one does Information Theoretic, a terminology that we will also em- not exist then the two concepts are maximally dissimilar, ploy in the present paper. The following section describes otherwise the shared information is equal to the IC value of our method of deriving IC values for existing concepts in the MSCA. Formally, semantic similarity is defined as: WordNet (Miller et al., 1990) along with the assumptionsmade and its formal definition. Section 4. presents the ex- perimental setup and a discussion of the results obtained evaluating our metric against human ratings of similarity.
where S(c1, c2) is the set of concepts that subsume c1 and When analyzing our results we also consider alternative approaches (i.e. non-information theoretic) in order to ex- Another information theoretic similarity metric that haustively evaluate our metric. In section 5. we suggest used the same notion of IC was that of (Lin, 1998). His how this similarity metric may be used for evaluating cre- ative recategorizations, possible extensions that may facil-itate the assessment of analogical similarity according to ”The similarity between A and B is measured the WordNet ontology are given in section 6. Comments by the ratio between the amount of information regarding our similarity metric will conclude this paper.
needed to state the commonality of A and B and the information needed to fully describe what A Formally the above definition may be expressed by: (icres(c1) + icres(c2)) (Jiang and Conrath, 1998) also continued on in the in- formation theoretic vein and suggested a new measure ofsemantic distance (if we consider the opposite1 of the dis- tance we obtain a measure of similarity) that combined the edge-based counting method with IC serving as a decisionfactor. Their model takes into consideration several other Figure 1: An example of multiple inheritance in the up- factors such as local density, node depth and link type, but per taxonomy of WordNet. ic and hc stand for Information
for the purpose of this paper we will only consider the case2 Content and Hyponym Count respectively.
where node depth is ignored and link type and local densityboth have a weight of 1. In this special case, the distance are leaf nodes are the most specified in the taxonomy so the information they express is maximal. In other words we express the IC value of a WordNet concept as a function of jcn(c1, c2) = (icres(c1)+icres(c2))−2×simres(c1, c2) Both Lin’s and Jiang’s formulation correct a problem existent with Resnik’s similarity metric; if one were to cal- res(c1, c1) one would not obtain the maximal similarity value, but instead the value given by icres(c1)3.
where the function hypo returns the number of hyponyms This problem is corrected in both subsequent formulations, of a given concept and maxwn is a constant that is set to yielding that simlin(c1, c1) is maximal and distjcn(c1, c1) the maximum number of concepts that exist in the taxon- omy4. The denominator, which is equivalent to the value ofthe most informative concept, serves as normalizing factor Information Content in WordNet
in that it assures that IC values are in [0, ., 1]. The above As was made clear in the previous section, IC is ob- formulation guarantees that IC decreases monotonically as tained through statistical analysis of corpora, from where we transverse from the leaf nodes to the root nodes as can probabilities of concepts occurring are inferred. Statistical be observed in figure 1. Moreover, the IC of the imaginary analysis has been receiving much attention and has proved top node of WordNet would yield an information content to be very valuable in several NLP tasks (Manning and Sch¨utze, 1999). We feel that WordNet can also be used as As result of multiple inheritance in some of WordNet’s a statistical resource with no need for external ones. More- concepts, caution must be taken so that each distinct hy- over, we argue that the WordNet taxonomy may be inno- ponym is considered only once. Consider again the situ- vatively exploited to produce the IC values needed for SS ation in figure 1, the concept artifact is an immediate hy- ponym of whole and object. Since whole is also a hyponym Our method of obtaining IC values rests on the assump- of object we must not consider the hyponyms of artifact tion that the taxonomic structure of WordNet is organized twice when calculating the number of hyponyms of object.
in a meaningful and structured way, where concepts with Obviously, this metric gives the same score to all leaf many hyponyms convey less information than concepts that nodes in the taxonomy regardless of their overall depth. As are leaves. We argue that the more hyponyms a concept has a consequence of this, concepts such as blue sky and moun- the less information it expresses, otherwise there would be tain rose both yield a maximum IC value of 1 despite one no need to further differentiate it. Likewise, concepts that being at a two link depth and the other at a nine link depthin the taxonomy, which is in accordance with our initial as- 1Note that we avoid using the word inverse which may be mis- sumption. However, some counter examples do exist that leading. If one were to simply mathematically inverse the distance disagree with the assumption; take the concept anything this would alter the magnitude of the resulting correlation coeffi- which is a leaf node thus yielding maximum IC. Qualita- cient. Suppose w1 and w2 represent the same concept hence have tively analyzing the amount of information conveyed by a semantic distance of 0, consider also that between w3 and w4 this concept may lead us to question the score given by our there is a distance of 1. If one were to consider the mathemati- metric which indeed seems to over exaggerate. But yet an- cal inverse function this would profoundly alter the magnitude of other perspective may lead us to ask: ”Why weren’t any comparison. In the distance scenario we have a difference of 1 nodes considered as hyponyms of anything?” Whatever the between the two pairs; in the similarity scenario we obtain a dif-ference of infinity between the two.
answer may be, we must recognize that certain commit- 2Which is also the most widely observed configuration in the ments had to be made by the designers of WordNet and 3Note that the MSCA that subsumes c There are 79689 noun concepts in WordNet 2.0.
that these may not always match our present needs. Irre- • F — The results obtained using the independent im- spective of this fact, in some NLP tasks like Information plementation of the adapted gloss overlap measure.
Retrieval where SS is essential, we will find that words like anything, nothing , something, . which yield exaggerated G — The results obtained using the independent im- IC scores are frequently stored in stop word lists and are ig- nored, which will somewhat attenuate these apparent con- • H — The results obtained using the independent im- Empirical Studies
• I — The results obtained using the independent imple- In order to evaluate our IC metric we decided to use the three formulations of SS presented in section 2. and substi- • J — The results obtained using the independent imple- tuted Resnik’s IC metric with the one presented in equation 6. In accordance with previous research, we evaluated theresults by correlating our similarity scores with that of hu- • K — The results obtained using our implementation man judgments provided by (Miller and Charles, 1991). In their study, 38 undergraduate subjects were given 30 pairs • L — The results obtained using our implementation of of nouns and were asked to rate similarity of meaning for each pair on a scale from 0 (no similarity) to 4 (perfect syn-onymy). The average rating for each pair represents a good • M — The results obtained using our implementation estimate of how similar the two words are.
In order to make fair comparisons we decided to use a independent software package that would calculate similar- It should be noted that in two of the configurations, ity values using previously established strategies while al- namely E and G, two word pairs were not considered in the lowing the use of WordNet 2.0. One freely available pack- correlation calculation. This is due to the fact that SemCor, age is that of Siddharth Patwardhan and Ted Pederson5; a small portion of the Brown Corpus, was used in obtaining which implement semantic relatedness measures described the concept frequencies to calculate the IC values. Sem- by (Leacock and Chodorow, 1998), (Jiang and Conrath, Cor is a relatively small sized corpus which contains about 1998), (Resnik, 1995), (Lin, 1998), (Hirst and St-Onge, 25% of the existing nouns in WordNet. The word crane 1998), (Wu and Palmer, 1994) and the adapted gloss over- (nor none of its hyponyms) that appear twice in the Miller lap measure by (Banerjee and Pedersen, 2003). Despite dataset does not appear in the corpus, thus no IC value may our focus being on SS, a special case of Semantic Relat- derived for the word. Due to this fact we decided to ignore edness, we decided to also evaluate how all of these algo- the entries that would need these values in their assessment rithms would judge the similarity of the 30 pairs of words and calculated correlation without considering them.
using WordNet 2.0. In addition to these we also used La- One last observation regarding our implementations tent Semantic Analysis (Landauer et al., 1998) to perform must be made before we discuss the results. Using Resnik’s similarity judgments by means of a web interface available and Lin’s formulas yields results in [0, ., 1] where 1 is max- imum similarity and 0 corresponds to no similarity whatso- Table 4.1. presents the similarity values obtained with ever. However, Jiang and Conrath’s measure is a measure the chosen algorithms and their correlation factor with hu- of semantic distance, in order to maintain the coherency of man judgments. Each of the capital letters heading each our implementations we decided to apply a linear transfor- column represents a different semantic relatedness algo- mation on every distance value in order to obtain a similar- rithm. The columns are organized in following manner: ity value7. Yet this transformation will only yield similar-ity values instead of distance, so normalization factor was • A — The data gathered by Miller and Charles Regard- also required in order to constrain the output to values to [0, ., 1]. The resulting formulation is: • B — The results obtained using the independent im- wn(c1) + icwn(c2) − 2 × simres (c1, c2) jcn(c1, c2) = 1−( plementation of the Leacock Chodorow measure.
• C — The results obtained using the independent im- Note that simres corresponds to Resnik’s similarity func- plementation of the simple edge-counts measure.
tion but now accommodating our IC values.
• D — The results obtained using the independent im- Discussion of Results
plementation of the Hirst St. Onge measure.
Observing table 4.1. we see that the algorithms per- formed fairly well. Established algorithms for which there • E — The results obtained using the independent im- are published results regarding the Miller compilation ap- plementation of the Jiang Conrath measure.
pear to be the same. The results obtained using our IC 7This transformation will not change the magnitude of the re- sulting correlation coefficient, although its sign may change from 6The web interface can be accessed at http://lsa.colorado.edu/.
negative to positive (Jiang and Conrath, 1998).
values in the information theoretic formulas (K, L and M) • Low — the creative value of the recategorization is in seem to have outperformed their homologues (H, G and E), which suggests that the initial assumption concerning the Some examples from each of these groups are given in taxonomic structure of WordNet is correct. It should be noted that the maximum value obtained, using Jiang andConrath’s formulation, is very close to what (Resnik, 1999) Analogical Similarity
proposed as a computational upper bound. Reproducing Analogy is regarded as an important creative reasoning the experiment performed by Jiang and Conrath where they mechanism, as such we feel that extending our metric to removed the pair furnace — stove from their evaluation deal with analogical similarity is very appealing. Obvi- claiming that MCSA for the pair is not reasonable8, we ob- ously, a simple taxonomic metric will not be able to cap- ture some of the deep similarities of an analogical insight, Similarity in Creative Recategorization
but taxonomic cues do exist that may shed some light ona potential analogy. As suggested by (Veale, submitted Considering the high correlation value obtained with manuscript), WordNet defines seed as hyponym of repro- configuration M and the hierarchical nature of the metric ductive structure and egg as a hyponym of reproductive we believe that it is an ideal candidate to fulfill the role of σ cell. Reproduction is thus the unifying theme of the analogy presented in equation 1. As a starting point for the valida- {seed-plant; egg-bird}. The strict taxonomic similarity be- tion of the above hypothesis, we conducted an exploratory tween seed and egg is very low yielding a value of 0.37, as experiment in which we generate new recategorizations and their lowest common WordNet hypernym is the root node then assess their creative value by substituting σ in equation entity. However, if reproductive structure and reproductive 1 with the SS metric used in configuration M. The recate- cell are treated as equivalent by considering the average of gorizations are generated by a process dubbed Category
their IC values as the IC value of a hypothetical analogical Broadening (Veale, 2004).
pivot we obtain a value of 0.88. We feel this value indicates As an example of this process imagine we want to the analogical similarities between egg and seed.
broaden the WordNet category weapon. The members ofthis category can be enumerated by recursively visiting ev- Conclusion and Future Work
ery hyponym of the category, which will include knife, gun, Obviously, the use of such a small dataset does not al- artillery, pike, etc. But by traversing polysemy links as low us to be conclusive regarding the true correlation be- well as isa relations, additional prospective members can
tween computational approaches of SS and human judg- be reached and admitted on the basis of their functional po- ments of similarity. Nevertheless, when our IC metric is tential. Thus, the polysemy of knife causes not only dag- applied in previously established semantic similarity for- ger and bayonet but steak knife and scalpel to be visited.
mulations, we find a very motivating quislingism. One ma- Stretching category boundaries even further, we may gener- jor advantage of this approach is that it does not rely on cor- alize that all edge tools maybe considered weapons, thereby pora analysis, thus we avoid the sparse data problem which allowing scissors, ax, razor and all other sharp-edged tools was evident in these experiments when judging pairs that to be recognized as having weapon-like potential.
At the heart of the broadnening process is the use of pol- Future work will consist of a more thorough evaluation ysemy links. Since WordNet does not contain these links of our metric regarding both its literal facet and also its po- explicitly a patchwork of polysemy detectors are needed.
tential to evaluate creative recategorizations. Another as- As such we implemented the polysemy detectors presented pect that will also deserve our future attention is the appli- in (Mihalcea and Moldovan, 2001) and (Veale, 2004) to cation of our metric to other taxonomic knowledge bases find the needed facilitating links. The new domain point- (e.g. Gene Ontology), allowing us to conclude if our in- ers of WordNet 2.0 were also used; basically we consider tuition about IC is generalizable to other taxonomic re- that if two senses of the same word belong to same domain then they are polysemous. We then applied the broadeningprocess described above to the WordNet 2.0 noun hierarchy References
and divided the generated recategorizations into 3 groups Banerjee, Satanjeev and Ted Pedersen, 2003. Extended gloss overlaps as a measure of semantic relatedness. In • High — the creative value of the recategorization is in Proceedings of the Eighteenth International Joint Con- ference on Artificial Intelligence. Acapulco, Mexico.
Bento, Carlos and Amilcar Cardoso, 2004. Studying cre- Medium — the creative value of the recategorization ativity in ai. Fachbereich Kunstliche Intelligenz der is in [0.33, 0.66[.
Gesellschaft fur Informatik:45–46.
de Bono, E., 1994. Parallel Thinking. London: Viking 8We agree with their claim in that a more informative sub- sumer should have been chosen, but we also think that algorithms Gangemi, Aldo, Nicola Guarino, Claudio Masolo, Alessan- dealing with manually constructed knowledge bases must be ableto deal with these situations as they are inescapable. Fortunately, dro Oltramari, and Luc Schneider, 2002. Sweetening some research has emerged that looks for these inconsistencies al- ontologies with dolce. In Proceeding of the European lowing a restructure of the taxonomy ((Veale, 2003), (Gangemi Workshop on Knowledge Acquisition, Modeling, and Algorithm
Correlation
Table 1: Results obtained evaluating correlation with human judgments using several algorithms and WordNet 2.0.
plane ticket isa leave of absence smoking room isa hiding place Table 2: Some examples of creative recategorizations grouped by their creative value.
Hirst, Graeme and David St-Onge, 1998. Lexical chains similarity. In Proc. 15th International Conf. on Machine as representations of context for the detection and cor- Learning. Morgan Kaufmann, San Francisco, CA.
rection of malapropisms. In Christiane Fellbaum (ed.), Manning, Christopher D. and Hinrich Sch¨utze, 1999.
WordNet: An Electronic Lexical Database, chapter 13.
Foundations of statistical natural language processing.
Hutton, J., 1982. Aristotle’s Poetics. New York: Norton.
Mihalcea, Rada and Dan Moldovan, 2001. Ez.wordnet: Jiang, J. and D. Conrath, 1998. Semantic similarity based Principles for automatic generation of a coarse grained on corpus statistics and lexical taxonomy.
wordnet. In Proceedings of Flairs 2001.
Landauer, T. K., P. W. Foltz, and D. Laham, 1998. In- Miller, George, Richard Beckwith, Christiane Fellbaum, troduction to latent semantic analysis. Discourse Pro- Derek Gross, and Katherine J. Miller, 1990. Introduction to wordnet: an on-line lexical database. International Leacock, C. and M. Chodorow, 1998. Combining local Journal of Lexicography, 3(4):235 – 244.
context and wordnet similarity for word sense identifi- Miller, George and W.G. Charles, 1991. Contextual cor- cation. In Christiane Fellbaum (ed.), WordNet: An Elec- relates of semantic similarity. Language and Cognitive tronic Lexical Database. MIT Press, pages 265–283.
Lenat, Douglas B. and R. V. Guha, 1990. Building Large Resnik, Philip, 1995. Using information content to evaluate Knowledge-Based Systems: Representation and Infer- semantic similarity in a taxonomy. In IJCAI.
Resnik, Philip, 1999. Semantic similarity in a taxonomy: An information-based measure and its application to Lin, Dekang, 1998. An information-theoretic definition of problems of ambiguity in natural language. Journal of Artificial Intelligence Research, 11:95–130.
Shannon, C.E., 1948. A mathematical theory of communi- cation. Bell System Technical Journal, 27:379–423 and623–656.
Torrance, E. P., 1990. The Torrance Tests of Creative Thinking. Illinois: Bensonville.
Veale, Tony, 2003. The analogical thesaurus: An emerg- ing application at the juncture of lexical metaphor andinformation retrieval. In Proceedings International Con-ference on Innovative Applications of Artificial Intelli-gence.
Veale, Tony, 2004. Pathways to creativity in lexical ontolo- gies. In Proceedings of the 2nd Global WordNet Confer-ence.
Wiggins, Geraint, 2003. Categorizing creative systems. In Proceedings of 3rd Workshop on Creative Systems.
Wu, Z. and M. Palmer, 1994. Verb semantics and lexical

Source: http://eden.dei.uc.pt/~nseco/LREC_WS.pdf