Using Ruby with Erb and YAML to build dialogs (long w/code).

[aGorilla]

Well, I wanted to try this out, and I finally got a chance to give it a shot, so here it is.

I'm not sure what I think of it yet, so I'd like to hear from others.

The first two files are the Ruby script, and the Erb template that gets applied to the YAML data.  The last two files are the Input/Output files, they're the ones to look at.

The ruby script:
MB_Dialogs.rb

Code: [Select]

require 'erb'
require 'yaml'
include YAML

unless ARGV[0]
  puts '', "Please provide a filename."
  system("pause")
  exit
end

source_file  = ARGV[0]
data = YAML.load( File.read(source_file) )

if data['file']
  lines = []
  File.open( "C:/Work/active/shell/mvCompiler/BIN/MB_Dialogs.erb" ) do |fh|
    erb = ERB.new( fh.read, nil, ">" )
    lines += erb.result( binding ).split(10.chr) # why just 10?
  end
  File.open(data['file'], 'w') {|f| f.puts lines}
end

The erb template file:
MB_Dialogs.erb

Code: [Select]

<% data['imports'].each do |import| %>from <%= import.split(' ')[0].strip %> import <%= import.split(' ')[1].strip %>

<% end %>

<%= data['name'] %> = [
<% data['dialogs'].each do |dialog| %> 
<% if dialog['actor'] %>
  [<%= dialog['actor'] %>, "<%= dialog['start'] %>", <% unless dialog['pre'] %>[],
<% end %>
<% if dialog['pre'] %>[
<% dialog['pre'].each do |cond| %>
    (<%= cond %>),
<% end %>
  ],
<% end %>
    "<%= dialog['text'] %>",
  "<%= dialog['end'] %>", <% unless dialog['post'] %>[]],<% end %>
<% if dialog['post'] %>[
<% dialog['post'].each do |cond| %>
    (<%= cond %>),
<% end %>  ]],<% end %>
<% end %>
<% end %>
]
The YAML file (INPUT):
dialog09_map_talks.yaml

Code: [Select]

## MAP TALKS

---
  name: dialog_map_talks
  file: dialog08_map_talks_TEST.py
  imports:
    - header_dialogs        *
    - header_operations     *
    - header_parties        *
    - header_item_modifiers *
    - header_skills         *
    - header_triggers       *
    - ID_troops             *
    - ID_party_templates    *
    - module_constants      *
  dialogs:
    -
      actor: party_tpl|pt_peasant
      start: start
      pre:
      text:  Greetings traveller.
      end:   peasant_talk_1
      post:  play_sound,"snd_encounter_farmers"
    -
      actor: party_tpl|pt_peasant|plyr
      start: peasant_talk_1
      pre:   eq,"$quest_accepted_zendar_river_pirates"
      text:  Greetings to you too.
      end:   close_window
      post:  assign, "$g_leave_encounter",1
    -
      actor: party_tpl|pt_peasant|plyr
      start: peasant_talk_1
      pre:
        -    neq, "$quest_accepted_zendar_river_pirates"
        -    eq, "$peasant_misunderstanding_said"
      text:  I have been charged with hunting down Shadow Knights in this area...
      end:   peasant_talk_2
      post:  assign,"$peasant_misunderstanding_said",1
    -
      actor: party_tpl|pt_peasant|plyr
      start: peasant_talk_1
      pre:
        -    neq,"$quest_accepted_zendar_river_pirates
        -    neq,"$peasant_misunderstanding_said"
      text:  Greetings. I am hunting Shadow Knights. Have you seen any around here?
      end:   peasant_talk_2b
      post: 

The results (OUTPUT):
dialog08_map_talks_TEST.py

Code: [Select]

from header_dialogs import *
from header_operations import *
from header_parties import *
from header_item_modifiers import *
from header_skills import *
from header_triggers import *
from ID_troops import *
from ID_party_templates import *
from module_constants import *

dialog_map_talks = [
 
  [party_tpl|pt_peasant, "start", [],
    "Greetings traveller.",
  "peasant_talk_1", [
    (play_sound,"snd_encounter_farmers"),
  ]],
 
  [party_tpl|pt_peasant|plyr, "peasant_talk_1", [
    (eq,"$quest_accepted_zendar_river_pirates"),
  ],
    "Greetings to you too.",
  "close_window", [
    (assign, "$g_leave_encounter",1),
  ]],
 
  [party_tpl|pt_peasant|plyr, "peasant_talk_1", [
    (neq, "$quest_accepted_zendar_river_pirates"),
    (eq, "$peasant_misunderstanding_said"),
  ],
    "I have been charged with hunting down Shadow Knights in this area...",
  "peasant_talk_2", [
    (assign,"$peasant_misunderstanding_said",1),
  ]],
 
  [party_tpl|pt_peasant|plyr, "peasant_talk_1", [
    (neq,"$quest_accepted_zendar_river_pirates),
    (neq,"$peasant_misunderstanding_said"),
  ],
    "Greetings. I am hunting Shadow Knights. Have you seen any around here?",
  "peasant_talk_2b", []],
 
]


[Neophyte]

My main gripe with the current dialog-editing isn't the format of the file (although what you're presenting here is clearly an improvemenet), but rather the *organization* of that file, which your suggestion unfortunately doesn't do much about.

In my head a dialog (as in, a set of statements/responses between 2 parties) is clearly a tree with each node being a statement by one of the people involved, and possible responses as child-nodes of that statement-node. The dialog-system, though, requires that dialog-statements be listed in a linear fashion, which is fairly un-intuitive (to me at least).

For instance, a simple dialog might look like this:

Code: [Select]

                   Party:1
                    /    \
                   /      \
            Player:1  Player:2
             /   \           \
            /     \           \
       Party:2  Party:3  <Done>
        /           \           
       /             \       
  Player:3      <Goto Party:1> 
      |
      |
  <Done>
(Each node here is represented by the party making the statement, and a number identifying the statement, so [Player:1] means some statement "1" made by the player).


I've idly toyed with some alternative ways to represent dialogs that more closely fits this sort of tree-structure, but haven't been able to come up with something I was really happy with (some form of XML-structure, for instance, is fairly representative, but I couldn't find a satisfactory way to represent the fact that dialogs have "loops" in them, where the "state" of the dialog returns to some previous statement in the dialog).


I'm going to take this opportunity, though, to ask you (and anyone else, for that matter):
If you could code a dialog any way you wanted to, with any sort of structure, use of keywords, etc, you could possibly want, then what would that look like?
Don't limit yourself to ways which could be represented in a language you know, or that exists, just imagine that you could specify a dialog any way you felt comfortable with, and then someone would make that language for you.

(This isn't just an intellectual excercise, btw, I'm currently working on a compiler-framework for m&b which necessarily includes a compiler and language-design to go with it, but I'm not really getting anywhere with dialog-coding).

[aGorilla]

Although my first little experiment does little more than change the format, I did have some other ideas in mind.  For one thing, I'd like to see some validation - make sure that each end 'state' has a corresponding dialog block.  I'd also like for it to be able to spot unused blocks.

I"m under the impression that there are times that you have to put some dialog in the wrong order - ie: to satisfy the 'first one found' rule.  I could be wrong about this (I'm still a noob), but if I'm not, it would be nice if the app could figure that out, and do it for you.

As far as how would I like to edit dialog, I'm inclined to want some form of a GUI.  Doesn't matter if that takes the form of a desktop app, or an AJAX enabled web app, but it seems you need some form of 'live' system.  Something that's aware of the potential 'actors', and the potential 'states'.

Aside from a live editor, I'd be inclined to go with something that looks more like a language than a specification.   Although you warned against it, I'm inclined to go 'with a language I know', if for no other reason, than... well... I know it.

Using Ruby as an example (or many other languages for that matter), you could use if/else/unless for most dialogs, and case statements for longer menus.  I'm already used to reading that code, and it's actually far more English-like than either, the current python format, or my YAML format.

If you like, I can try to work up an example, but I imagine you get the general idea.

[fujiwara]

Just to add my two cents:

Language is difficult. Otherwise, we would have found a way to program computers with natural language skills already. My beef with the current system is the "first in the list" approach taken by armagan, which certainly isn't the most optimal, but it is the simplest to implement. You have an array of values, and you test each one until all conditions evaluate True. The current system of dialog programming is a result of that approach, and unless armagan decides on a different method, I think any system will by nature, be rather cumbersome. I think Neophyte's XML tree probably would work best, and just acknowledge the fact that it doesn't represent loops well. That would require a 3D tree. I guess if the sky were the limit, I would set up each dialog as a class, and use the methods to link them together, something like (in Python):

Code: [Select]

class DialogInnkeeperStart:
        def __init__():
                self.text = "Good day, sir. How can I help you"
                self.dlgLink01 = DialogInnkeeperEnd()
                self.dlgLink02 = DialogInnkeeperSleep()
                self.plyrChoice01 = "I would like to spend the night."
                self.plyrChoice02 = "Never mind. Good bye."
                self.links = [self.dlgLink01,self.dlgLink02]
                self.choices = [self.plyrChoice01,self.plyrChoice02]

class DialogInnkeeperSleep:
        def __init__():
                self.text = "It will be 10 gold, sir."
                self.dlgLink01 = DialogInnkeeperSleep2()
                self.dlgLink02 = DialogInnkeeperNoMoney()
                self.dlgLink03 = DialogInnkeeperEnd()
                self.plyrChoice01 = "Here is your money."
                self.plyrChoice02 = "I don't have that much."
                self.plyrChoice03 = "Never mind."

        def logicalTest(self):
                current_gold = getPlayerGold()
                if current_gold >= 10:
                        self.links = [self.dlgLink01,self.dlgLink03]
                        self.choices = [self.plyrChoice01,self.plyrChoice03]
                        self.actions = [modTroopGold(player,-10)]
                else:
                        self.links = [self.dlgLink02,self.dlgLink03]
                        self.choices = [self.plyrChoice02,self.plyrChoice03]

Of course, the problem with a system like this is writing the complier to convert it to M&B code What's left out here is the function definition for things like getPlayerGold() and modTroopGold(trp,num)

[Neophyte]

I've been thinking a little more about it, and I do think a structured approach is the best (at least for my brain ).
However, I do find XML to be too verbose to be extremely useful, so maybe a more lightweight approach would be more useful.
For instance, something along these lines:

Code: [Select]

dialog = {
  name = "testdialog"
  target = "trp_foobar"
 
  statement = {
    by = target
    text = "Hello there, this is some generic text ... blah blah"
  }
   
  statement = {
    by = player
    text = "Why hello there."
  }

  statement = {
    id = "foobar"
    by = target
    text = "What can I do for you?"
  }

  options = {

    option = {
      condition = { (store_troop_gold,reg(1),trp_player), (gt,reg(1),1000) }

      statement = {
        by = player
        text = "I'd like to give you a thousand dinars."
      }

      statement = {
        by = target
        text = "Why, thank you."
        after = { (troop_remove_gold,"trp_player",1000) }
        next = "foobar"
      }
    }

    option = {
      statement = {
        by = player
        text = "Nothing, nevermind."
        next = "close_window"
      }
    }

  }

}       

Which would translate into this:

Code: [Select]

[trp_foobar,"start",[], "Hello there, this is some generic text ... blah blah", "testdialog2", []],
[anyone|player, "testdialog2",[], "Why hello there.", "testdialog3", []],
[anyone, "testdialog3", [], "What can I do for you?", "testdialog4", []],
[anyone|player, "testdialog4",
[(store_troop_gold,reg(1),"trp_player"),(gt,reg(1),1000)], "I'd like to give you a thousand dinars", "testdialog5", []],
[anyone|player, "testdialog4", [], "Nothing, nevermind.", "close_window", []],
[anyone, "testdialog5", [], "Why, thank you.", "testdialog3", [(troop_remove_goold,"trp_player",1000)]],

The idea being that a sequence of simple statements back and forth are coded as a sequence of "statement = { ... }" entries, and whenever multiple options are available to either person in the dialog it's represented by

Code: [Select]

options = {
 option = {
  condition = { <the condition for this option> }
  statement = { ... }
  statement = { ... }
 }
 option = {
  <no condition for this, so always available>
  statement = { ... }
  statement = { ... }
 }
}

Only the "by = ... " and "text = ..." values are required for any statements, the rest can be inferred if necessary.
If a dialog needs to "loop back" to a previous statement then that statement needs an "id = ..." entry, and the statement that loops back has a "next = <id of other statement>" entry.

One of the main advantages I see to doing something like this is that the code is a lot less error-prone that manually fiddling with the dialogs-files, as the structure of the dialog is a lot more obvious from looking at the code, and the fact that you don't need to connect start-states and end-states unless your dialog loops makes things a lot easier.

[aGorilla]

The more I look at that, the more I like it.  At first, I saw things I didn't like, but after a little thought, even those became features instead of bugs.  To top it off, it's probably pretty easy to parse.

The one change I would suggest would be to change 'options' to 'menu'.  It would make it more obvious what it was, and would avoid any confusion between 'option' and 'options'.

The one question I still have:  are there times when the dialog has to be out of sequence - to satisfy the 'find first' approach of dialog files?

[Neophyte]

options -> menu sounds like a very sensible change, well spotted.

I would be interested in any major or minor quibbles anyone has about anything, as it was quite frankly thrown together reasonably quickly and I would be very surprised if there aren't more things in there I haven't thought of, or that could be improved.


As for your "out of sequence" question, I haven't fiddled enough with dialogs myself to say much useful about it, I'm afraid. I'm sure there are others here with much more experience than me who can answer that.
As far as my own limited experience is concerned the most noticable issue with it is that you can't just throw in new dialog-options any old place (especially if you use "anyone" as the troop-type rather than the specific type), but have to make sure the "catch-all" options are at the end of any given sequence. I'm sure there are more complex cases that I just haven't run into yet, though.

[aGorilla]

Some more thoughts on the above example.

Inside of the options/menu block, shouldn't the 'condition' clause be part of the first statement block?

At first, the 'Why, thank you.' didn't seem to belong in there, but I see it's nested so you can have the followup without having to name it.

Some more thoughts on naming - Part of the problem, in my mind, is that so many of these can overlap.  The current words 'dialog', 'statement', and 'text' are pretty much interchangeable, as they should be.   The way the dialog is defined, with two things 'before', and two things 'after', also adds to the ambiguity when it comes to naming.

In your example, I would drop the quotes around 'name', and 'target'.
I'm also inclined to drop the braces around the conditions - it might also be helpful to allow multi-line conditions.

here's a quick thesaurus dump...

dialog - chat - conversation - talk - discourse - speech
condition/after - if/and/then - conditions/consequences - pre/post - before/after
next - after - following - followup - reply - response
statement - say - saying - said - verse - line - express - expression - expressed - words
target - actor - player - character

atm, I'm not quite sure what I would pick from the above.  I'm pretty happy with 'dialog', 'menu', 'option', and 'by'.  I think I'd go with if/and/then for pre and post conditions, using 'and' for multiple line conditions.  I also like 'reply' over 'next', it seems to fit well, and it even seems to work with 'close_window' (at least for me).

I might just take a shot at parsing this with Ruby.  I think it's an improvement over the current system, and it keeps the 'clarity' that I was looking for when I tried the YAML file.

[Neophyte]

The naming wasn't tremendously thought through, but the thinking I did was something like this:

- A "statement" is a single utterance by one of the people involved.
- A "dialog" is a sequence of "statements".  ("dialogue: a conversation between two persons", according to some web-dictionary or other).
- At certain points in a dialog one of the parties have multiple choices of statements, represented by what I initially called "options". Each "option" would start a separate sequence of statements, with each option possibly having some pre-conditions that have to be satisfied for that option to be a "legal" choice.

The reason the conditional in the "options" part is in the options and not in the statement is that the option actually starts a series of statements (player: "I'd like to give you money", target: "Why, thank you"), so it seemed more natural to me to put the condition in the "series of statements" part, rather than the "first statement in a series of statements" portion.

So, as it stands, I'm happy with the naming of "dialog", "statement", and "option". But I agree that what I called "options" in the original suggestion would be better named "menu" (although it seems a *little* odd that an NPC in a conversation is given a menu of possible responses, but I still think menu is better).
Also, renaming "next" to "response" seems sensible to me.


The reason for the braces around the conditions and "after" portions is precisely so that you can have multi-line code in there without making parsing the code needlessly complicated (it's a *lot* easier to parse multi-line data if the start and end of the block is clearly and cleanly separated from the actual data, by for instance sticking braces around it ).

The use (and non-use) of quotation-marks around certain data-entries is not consistent, which it should be.
I suspect the most intuitive is to have quotation-marks around the texts only, and skip it for everything else. If I were to code a parser for this sort of thing I'd probably make it not care one way or the other (probably by simply stripping away quotation-marks on everything it reads).

The way the dialog is defined, with two things 'before', and two things 'after', also adds to the ambiguity when it comes to naming.

I'm not sure what you're referring to here, to be honest, could you re-phrase it perhaps?


Really appreciate the feedback, btw.

[fujiwara]

Man, I like that. Much better than my approach. My other beef with the current dialog system is the lack of identifiers for each dialog, but you seem to have dealt with that issue very nicely. Kudos!

[aGorilla]

Yeah, I figured it was a quick shot at it.  Running things through a thesaurus is a habit of mine.

The placement of "Why, thank you" seemed off at first, but after pondering it a bit, it made a lot of sense.

I suggested "menu" while thinking as a programmer.  That is what you're building.  I still like it, but I'll toss out "choose" as an alternative.

Yeah, I had considered that you were using the braces for parsing.  Could use 'lines that begin with "and"', but either way, it works.

I completely agree on the quotes.  Just keep them for the text.  It makes sense there, and helps to remind you not to use double quotes in your text.

The way the dialog is defined, with two things 'before', and two things 'after', also adds to the ambiguity when it comes to naming.

I'm not sure what you're referring to here, to be honest, could you re-phrase it perhaps?

I'm talking about the layout for dialogs...

Code: [Select]

  this_ID
  pre_condition
  text
  next_ID
  post_condition

The fact that the text is surounded by two very similiar things.  In my YAML example, I used start/pre and end/post - and was torn/confused while I named them.