* ---------------------------------------------------------------------- * Copyright (c) 2014 Scott Furry * * This file is part of Freedroid * * Freedroid is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * Freedroid is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with Freedroid; see the file COPYING. If not, write to the * Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, * MA 02111-1307 USA * * ---------------------------------------------------------------------- Freedroid RPG - wiki parsing README ---------- Rationale: ---------- Information on the wiki pages can sometimes get out of date or become inconsistent with the current release. Wiki page editing/updating can sometimes become a major chore espically with major changes to the code base. ------------ Description: ------------ Wiki Parsing automates the process of parsing game source files and updating wiki pages containing game playing information. The wiki pages produced as a result include NPC/Character/Item information and Map of levels in game. ---------- Execution: ---------- cd lua doc/wiki/scripts/wiki_parse.lua [-s] [-l] [-v | -vv] switches: s enable sandbox output - disables some wiki text conversions - output can be pasted into wiki sandbox for testing (see assumptions: OUTPUT NOT HUMAN READABLE below) l produce a list of file links - saved as FDRPG root/doc/wiki/wiki.d/output_images.txt (see assumptions: PATH STRUCTURE - FILE NAMES below) v verbose output - produce verbose output to stdout - note large scrollback produced - prints out basic information about parsing vv double verbose output - produce extreme verbose output to stdout - implies verbose output - note HUGE scrollback produced - prints out basic information about parsing - prints out detailed module table contents RECOMMEND: pipe this output to file for easier review e.g. lua doc/wiki/scripts/wiki_parse.lua [-v | -vv] > [path/]filename -- output path should be used to prevent writing garbage to FDRPG root -- suggest doc/wiki/wiki.d for path as .gitignore is set to ignore *.txt files NOTE: Wiki Parsing is intended to be used by the FDRPG buildbot as part of build process. INPUT All scripts necessary for wiki parsing are located at FDRPG root/doc/wiki/scripts OUTPUT Wiki Parsing will save files to the following locations: wiki pages for uploading FRRPG root/doc/wiki/wiki.d/Doc.* images for wiki pages FRRPG root/doc/wiki/uploads These locations are then to be rsync'd with the FDRPG wiki site. LUA MODULE LOADING wiki_parse.lua processes the following FDRPG data files using the given lua code modules droid_archetypes.dat - modWP_Droid.lua events.dat - modWP_Events.lua item_specs.lua - modWP_Items.lua levels.dat - modWP_Levels.lua npc_specs.lua - modWP_NPC.lua ReturnOfTux.droids - modWP_ROTD.lua quests.dat - modWP_Quests For each module, the sequence used is: - 'require' module (with path - see Assumptions:PATH STRUCTURE below) - call module's "ProcessData" function -> process source files into data structs - call module's WikiWrite function -> convert data structures to wiki pages ## NOTE: some code modules may not have a WikiWrite function - but the module is needed to parse information. The produced data is used by other modules for wiki pages. ## NOTE: process modWP_Levels and modWP_NPC last (in that order) due to dependencies on other modules FDRPG LUA DATA FILES (i.e. item_specs.lua and npc_specs.lua ) FDRPG data files in lua format are parsed differently than plain text files. Functions used to extract data must be in the lua global space or parsing will fail. Also modWPCommon.Process.FileToChunk() function is used to removed gettext markers before processing as lua's dofile() will fail if gettext markers are present. Once the lua data files is loaded into memory as a "chunk", variables and structures in the data file can be accessed directly by the wiki parsing script. ------------ Assumptions: ------------ BUILDBOT Files from previous iterations/executions are removed by the buildbot. PLATFORM It is assumed that wiki_parsing.lua and its modules are being executed on a Linux-based platform. Also, lua 5.2 is required. No other third-party lua modules/dependencies are used. The script also executes a bash script for some file handling processes. PAGE HISTORY History of pmwiki page edits on auto-generated pages will not be retained between builds. The process to keep this history is specific to pmwiki and not practical to integrate or emulate. A warning is posted on auto-generated pages to users about the lack of edit/history retension. Changes to the layout of auto-generated wiki pages should go through the Review Board process. PATH STRUCTURE - FILE NAMES Lua is not very aware of path structures and file names. The paths and file names used to find lua modules and FDRPG data files had to be hardcoded into the wiki parsing source. Any changes made to the overall FDRPG project structure will need to be pushed into wiki parsing. When executing wiki_parsing, it is assumed that the current working directory is set to the FDRPG root. Any other location will cause the script to fail and exit. DATA FILES - TEXT MARKERS Data files make frequent use of text markers (e.g. levels.dat "Levelnumber:") to identify data. Assumption is that these markers do not change. If the markers are changed, changes must be pushed into wiki parsing source. OUTPUT NOT HUMAN READABLE The output of wiki parsing is not meant to be human readable. The data written to file is intended to be used directly as a wiki page. If one wishes to view the data, perform the following (current for pmwiki): 1) pass the flag "-s" to wiki_parse.lua - this will set the value of modWPCommon.sandbox to "true" and disable some wiki conversions 2) open resulting doc/wiki/wiki.d/Doc.* file 3) copy everything after "text=" and before "time=" 4) open the page https://www.freedroid.org/pmwiki/pmwiki.php?n=Main.WikiSandbox?action=edit in your browser - FDRPG pmwiki sandbox page in edit mode 5) paste into the sandbox page text box 6) select "preview" button to view resulting wiki page ---------- Features: ---------- modWPCommon.lua function "WikifyText" replaces characters pmwiki considers "special". see - http://www.pmwiki.org/wiki/PmWiki/PageFileFormat "text=" description. Currently -> replace "%" with %25 <- must do this FIRST replace "<" with %3c replace "\n" with %0a modWPCommon.lua function "extractText" - built to remove the gettext markers and quotation marks from a string. This function is used frequently when extracting text data from FDRPG data files. Adding/Removing gettext markers to data in source files should not cause the wiki parse process to fail or produce bad results. .gitignore files are used to exclude images and Doc.* files. The images are copies of items already under git control. Also, wiki parsing can change the contents of Doc.* files frequently. Efforts have been made to centralize the definitions of wiki text markup to a single file should the decision be made to change wiki engines. This is a future proofing function.