7.3 Idiomatic natural language command and question interfaces

The everyday meaning of the term "idiomatic expression" involves a natural language phrase whose grammatical form follows typical language patterns but whose meaning is not determined sufficiently by the usual meanings of the individual words in the expression. Principally, there are idiomatic noun phrases ("an early hard winter"), and whole sentences which are idiomatic ("Leave no stone unturned.").

Here, we will use a looser definition of idiomatic expression, to mean, roughly, whatever phrase a typical user of the software in question might use to communicate with the running application. By an idiom we will mean a collection of grammar rules with a common meaning. There may be many special (idiomatic) ways to say the same thing, that is, to convey the same meaning.

For example, consider the blocks world of Section 2.19 . Imagine a user interface that takes user input in the form of typed commands and questions. A more useful interface would be one that would take spoken commands, but this section only considers inputs which have already been broken into a sequence of words, as considered in the previous section, 7.2.

We will consider two idioms, the first of which is a command, the second a question.

The first idiom is the command to bring about a particular arrangement of blocks. One way to specify this idiom is to give some scripts that the command interface must translate into the intended meaning. For example, ...

Command idiom c:
{Please} place < put>  {block} X on < onto > {block} Y, W on Z, ...
I want < would like>  X on Y, W on Z, ...
I want < would like>  you to put < place>  ...
Can < could>  < would>  you {please} put < place>  X on Y, ...
The scripts are indicated informally here. Braces {...} allow optional words and brackets < ...> indicated alternate words. In this example, the command idiom will be the collection of grammar rules which parse all of these scripted commands and the corresponding meaning will be the abstract command to bring about the arrangement of blocks described by such a command. Of course, the arrangement may be physically invalid, so the command may amount to an impossible implied action. In Section 8.3, we will graphically animate the command interface, and thereby supply a more concrete meaning for the abstract commands. Thus, the meaning of the original input actually gets "interpreted" twice: first by the idiom (the grammar rules) producing a formal meaning and then the formal meaning is interpreted by the simulator to determine how to carry out the abstract action. Physically valid commands will be carried out by rearranging the blocks as required by the intended meaning of the command; physically invalid, or impossible, commands will get some appropriate response from the system.

Here is a partial grammar for the command idiom

c(L) --> lead_in,arrange(L),end.

end --> ['.'] | ['?'].

lead_in --> please, place.
lead_in --> [i], [want] | [i], [would], [like], you_to_put.
lead_in --> ([can] | [could] | [would]), [you], please, place.

you_to_put --> [] | [you], [to], place.   %%% partially optional

please --> [] | [please].    %%% optional word

place --> [put] | [place].   %%% alternate words

arrange([ON]) --> on(ON).
arrange([ON|R]) --> on(ON), comma, arrange(R).

comma --> [','] | ['and'] | [','],[and].   %%% alternate words

on(on(X,Y)) --> block, [X], ([on] | [onto] | [on],[top],[of]), block, [Y].
on(on(X,table)) --> [X],([on] | [onto]), [the], [table].

block --> [] | [block].   %%% optional word

:- [read_line].

test_parser :- repeat,
               write('?? '), 
               read_line(X),
               ( c(F,X,[])   | q(F,X,[])  ),
               nl, write(X), nl, write(F), nl, fail.
Here is a sample parse for a command ...
?- test_parser.
?? Please put block a on top of block b, c on d, and f on the table.

[please,put,block,a,on,top,of,block,b,(,),c,on,d,9,0,and,f,on,the,table,.]
[on(a, b),on(c, d),on(f, table)]
?? ...
The first output [please,put,...] is the user input converted to a sequence of words, using 'read_line', and the second output '[on(a,b),...]' is the abstract meaning corresponding to the user input. In this case, it is just a list of 'on' relationships that could be the intented arrangement to be brought about as a result of the intended command.

Before we discuss a question idiom it should be explained that we will change the internal representation for the positioning of blocks. Previously, in Section 2.19, this was simply done with the 'on(-,-)' property. An animator for the blocks mover will be given in Section 8.3. In that program, blocks will be given a location 'location(Block,[X,Y])' relative to where they (currently) sit on the graphical drawing plane (X and Y giving the location of the upper left corner of a block). The 'on(-,-)' property is then defined in terms of 'location(-,[-,-])'. The blocks are drawn 5 units high.

 
   /*
   Cursor graphics for blocks animator
   
       positioning information
   
          [20,2 ]    [30,  ]   [40,  ]
          [20,7 ]    [30,  ]   [40,  ]
          [20,12]    [30,  ]   [40,  ]
          [20,17]    [30,  ]   [40,  ]
        -==============================-   < --- table
     
       The actions 'up', left', 'right', and 'down' will, in effect,
       move a block one position;  the animation of such a
       move will show movement for each screen cursor position.
   */
 
   free_spot_on_table([30,17]).
   free_spot_on_table([40,17]).
   
     %% initially
   location(a,[20,7]).
   location(b,[20,12]).
   location(c,[20,17]).
  
   on(A,table) :- location(A,[_,17]).
   on(A,B) :- B \== table,
              location(A,[X,YA]),
              location(B,[X,YB]),
              YB is YA + 5.
Now, let us add a simple script for a question idiom.
Question idiom q:
Which block is on top of X?
What is on top of X?
and a partial grammar for the question idiom ...
:- op(500, xfx, 'is_on_top_of').

  %%% the question q
q(X is_on_top_of A) --> [which],[block],[is],[on],[top],[of],[A],end.
q(X is_on_top_of A) --> [what],[is],[on],[top],[of],[A],end.

   %%% How to answer q
B is_on_top_of A :- location(A,[X,Y]),
                    Y1 is Y-5,           %% What's above?
                    location(B,[X,Y1]), !.
'Nothing' is_on_top_of A .

answer(X is_on_top_of A) :- call(X is_on_top_of A),
                            say([X,is,on,top,of,A]).

say([X|R]) :- write(X), write(' '), say(R).
say([]).
Here is a sample parse for a question ...
?- test_parser.
?? Which block is on top of b?
[which,block,is,on,top,of,b,?]
G1220 is_on_top_of b

?? What is on top of a?
[what,is,on,top,of,a,?]
G1136 is_on_top_of a

??..
Now, the sample program also shows how the question will be answered -- see Section 8.3. The abstract meaning of the question "Which block is on top of b?" is Q = 'G1220 is_on_top_of b'. The way that this question will be answered is to pose the Prolog query Q, an hope to bind the variable G1220 to whatever block is on top of b; then 'say' the corresponding answer. If there is no block on top of b, then the Q goal will fail, and in this case the answer will be 'Nothing', and say, "Nothing is on top of b".

For each exercise, formulate a reasonable script for the idiom, and specify a reasonable abstract meaning for the idiom. It will probably be very helpful to read section 8.3, and run the blocks animator yourself, before doing the exercises.

Exercise 7.3.1 Formulate a new question idiom, one of whose scripts is "What is block X sitting on?".

Exercise 7.3.2 Formulate a new question idiom, one of whose scripts is "Which blocks are on the table?".

Exercise 7.3.3 Formulate a new command idiom, one of whose scripts is "Put all of the blocks in a single pile.". This is an interesting idiom, because the abstract meaning is ambiguous -- that is, there could be more than one abstract meaning for this idiom. Assume that one abstract meaning is no better or worse than another; just generate one of them as "the" meaning.

Exercise 7.3.4 Formulate a new command idiom, one of whose scripts is "Put the block on top of X on top of block y." (or on the table). This will involve interpreting the definite description ("the ...") acurately.

Exercise 7.3.5 Formulate a new command idiom, one of whose scripts is "Put the highest block on top of block y." (or the table). This will involve interpreting the definite description ("the ...") acurately.

Exercise 7.3.6 Design a program that can convert command or question text-based "scripts" into an idiom (collection of grammar rules with common meaning). Presumably, the user could type the scripts and the meaning in a somewhat informal way, and the program would generate the Prolog logic grammar rules automatically. Try to devise a partially visual format for laying out the inputs. A modified version of this exercise, if you have voice recognition, is to build a vocal user interface (VUI) that takes vocal inputs and generates the command and question interface (assuming an appropriate way is found to designate the meanings).


Prolog Code for this section.
Prolog Tutorial Contents