| 1 |
* ---------------------------------------------------------------------- |
| 2 |
* Copyright (c) 2014 Scott Furry |
| 3 |
* |
| 4 |
* This file is part of Freedroid |
| 5 |
* |
| 6 |
* Freedroid is free software; you can redistribute it and/or modify |
| 7 |
* it under the terms of the GNU General Public License as published by |
| 8 |
* the Free Software Foundation; either version 2 of the License, or |
| 9 |
* (at your option) any later version. |
| 10 |
* |
| 11 |
* Freedroid is distributed in the hope that it will be useful, |
| 12 |
* but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 13 |
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 14 |
* GNU General Public License for more details. |
| 15 |
* |
| 16 |
* You should have received a copy of the GNU General Public License |
| 17 |
* along with Freedroid; see the file COPYING. If not, write to the |
| 18 |
* Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, |
| 19 |
* MA 02111-1307 USA |
| 20 |
* |
| 21 |
* ---------------------------------------------------------------------- |
| 22 |
Freedroid RPG - wiki parsing README |
| 23 |
---------- |
| 24 |
Rationale: |
| 25 |
---------- |
| 26 |
Information on the wiki pages can sometimes get out of date or become inconsistent |
| 27 |
with the current release. Wiki page editing/updating can sometimes become a major |
| 28 |
chore espically with major changes to the code base. |
| 29 |
|
| 30 |
------------ |
| 31 |
Description: |
| 32 |
------------ |
| 33 |
Wiki Parsing automates the process of parsing game source files and updating wiki |
| 34 |
pages containing game playing information. The wiki pages produced as a result |
| 35 |
include NPC/Character/Item information and Map of levels in game. |
| 36 |
|
| 37 |
---------- |
| 38 |
Execution: |
| 39 |
---------- |
| 40 |
cd <path to FDRPG root> |
| 41 |
lua doc/wiki/scripts/wiki_parse.lua [-s] [-l] [-v | -vv] |
| 42 |
switches: |
| 43 |
s enable sandbox output - disables some wiki text conversions |
| 44 |
- output can be pasted into wiki sandbox for testing |
| 45 |
(see assumptions: OUTPUT NOT HUMAN READABLE below) |
| 46 |
l produce a list of file links |
| 47 |
- saved as FDRPG root/doc/wiki/wiki.d/output_images.txt |
| 48 |
(see assumptions: PATH STRUCTURE - FILE NAMES below) |
| 49 |
v verbose output - produce verbose output to stdout |
| 50 |
- note large scrollback produced |
| 51 |
- prints out basic information about parsing |
| 52 |
vv double verbose output - produce extreme verbose output to stdout |
| 53 |
- implies verbose output |
| 54 |
- note HUGE scrollback produced |
| 55 |
- prints out basic information about parsing |
| 56 |
- prints out detailed module table contents |
| 57 |
RECOMMEND: pipe this output to file for easier review |
| 58 |
|
| 59 |
e.g. lua doc/wiki/scripts/wiki_parse.lua [-v | -vv] > [path/]filename |
| 60 |
-- output path should be used to prevent writing garbage to FDRPG root |
| 61 |
-- suggest doc/wiki/wiki.d for path as .gitignore is set to ignore *.txt files |
| 62 |
|
| 63 |
NOTE: |
| 64 |
Wiki Parsing is intended to be used by the FDRPG buildbot as part of build process. |
| 65 |
|
| 66 |
INPUT |
| 67 |
All scripts necessary for wiki parsing are located at FDRPG root/doc/wiki/scripts |
| 68 |
|
| 69 |
OUTPUT |
| 70 |
Wiki Parsing will save files to the following locations: |
| 71 |
wiki pages for uploading FRRPG root/doc/wiki/wiki.d/Doc.* |
| 72 |
images for wiki pages FRRPG root/doc/wiki/uploads |
| 73 |
|
| 74 |
These locations are then to be rsync'd with the FDRPG wiki site. |
| 75 |
|
| 76 |
LUA MODULE LOADING |
| 77 |
wiki_parse.lua processes the following FDRPG data files using the given lua code modules |
| 78 |
droid_archetypes.dat - modWP_Droid.lua |
| 79 |
events.dat - modWP_Events.lua |
| 80 |
item_specs.lua - modWP_Items.lua |
| 81 |
levels.dat - modWP_Levels.lua |
| 82 |
npc_specs.lua - modWP_NPC.lua |
| 83 |
ReturnOfTux.droids - modWP_ROTD.lua |
| 84 |
quests.dat - modWP_Quests |
| 85 |
|
| 86 |
For each module, the sequence used is: |
| 87 |
- 'require' module (with path - see Assumptions:PATH STRUCTURE below) |
| 88 |
- call module's "ProcessData" function -> process source files into data structs |
| 89 |
- call module's WikiWrite function -> convert data structures to wiki pages |
| 90 |
## NOTE: some code modules may not have a WikiWrite function - but the module is needed to parse information. |
| 91 |
The produced data is used by other modules for wiki pages. |
| 92 |
## NOTE: process modWP_Levels and modWP_NPC last (in that order) due to dependencies on other modules |
| 93 |
|
| 94 |
FDRPG LUA DATA FILES |
| 95 |
(i.e. item_specs.lua and npc_specs.lua ) |
| 96 |
FDRPG data files in lua format are parsed differently than plain text files. |
| 97 |
|
| 98 |
Functions used to extract data must be in the lua global space or parsing will fail. |
| 99 |
Also modWPCommon.Process.FileToChunk() function is used to removed gettext markers before processing as |
| 100 |
lua's dofile() will fail if gettext markers are present. |
| 101 |
|
| 102 |
Once the lua data files is loaded into memory as a "chunk", variables and structures in the data |
| 103 |
file can be accessed directly by the wiki parsing script. |
| 104 |
|
| 105 |
------------ |
| 106 |
Assumptions: |
| 107 |
------------ |
| 108 |
BUILDBOT |
| 109 |
Files from previous iterations/executions are removed by the buildbot. |
| 110 |
|
| 111 |
PLATFORM |
| 112 |
It is assumed that wiki_parsing.lua and its modules are being executed on a Linux-based platform. |
| 113 |
Also, lua 5.2 is required. No other third-party lua modules/dependencies are used. |
| 114 |
|
| 115 |
The script also executes a bash script for some file handling processes. |
| 116 |
|
| 117 |
PAGE HISTORY |
| 118 |
History of pmwiki page edits on auto-generated pages will not be retained between builds. The process |
| 119 |
to keep this history is specific to pmwiki and not practical to integrate or emulate. A warning |
| 120 |
is posted on auto-generated pages to users about the lack of edit/history retension. |
| 121 |
|
| 122 |
Changes to the layout of auto-generated wiki pages should go through the Review Board process. |
| 123 |
|
| 124 |
PATH STRUCTURE - FILE NAMES |
| 125 |
Lua is not very aware of path structures and file names. The paths and file names used to find |
| 126 |
lua modules and FDRPG data files had to be hardcoded into the wiki parsing source. Any changes made to |
| 127 |
the overall FDRPG project structure will need to be pushed into wiki parsing. |
| 128 |
|
| 129 |
When executing wiki_parsing, it is assumed that the current working directory is set to the FDRPG root. |
| 130 |
Any other location will cause the script to fail and exit. |
| 131 |
|
| 132 |
DATA FILES - TEXT MARKERS |
| 133 |
Data files make frequent use of text markers (e.g. levels.dat "Levelnumber:") to identify data. |
| 134 |
Assumption is that these markers do not change. If the markers are changed, changes must be |
| 135 |
pushed into wiki parsing source. |
| 136 |
|
| 137 |
OUTPUT NOT HUMAN READABLE |
| 138 |
The output of wiki parsing is not meant to be human readable. The data written to file is intended to |
| 139 |
be used directly as a wiki page. If one wishes to view the data, perform the following (current for pmwiki): |
| 140 |
1) pass the flag "-s" to wiki_parse.lua |
| 141 |
- this will set the value of modWPCommon.sandbox to "true" and disable some wiki conversions |
| 142 |
2) open resulting doc/wiki/wiki.d/Doc.* file |
| 143 |
3) copy everything after "text=" and before "time=" |
| 144 |
4) open the page https://www.freedroid.org/pmwiki/pmwiki.php?n=Main.WikiSandbox?action=edit in your browser |
| 145 |
- FDRPG pmwiki sandbox page in edit mode |
| 146 |
5) paste into the sandbox page text box |
| 147 |
6) select "preview" button to view resulting wiki page |
| 148 |
|
| 149 |
---------- |
| 150 |
Features: |
| 151 |
---------- |
| 152 |
modWPCommon.lua function "WikifyText" replaces characters pmwiki considers "special". |
| 153 |
see - http://www.pmwiki.org/wiki/PmWiki/PageFileFormat "text=" description. |
| 154 |
Currently -> replace "%" with %25 <- must do this FIRST |
| 155 |
replace "<" with %3c |
| 156 |
replace "\n" with %0a |
| 157 |
|
| 158 |
modWPCommon.lua function "extractText" - built to remove the gettext markers and quotation marks |
| 159 |
from a string. This function is used frequently when extracting text data from FDRPG data files. |
| 160 |
Adding/Removing gettext markers to data in source files should not cause the wiki parse process to |
| 161 |
fail or produce bad results. |
| 162 |
|
| 163 |
.gitignore files are used to exclude images and Doc.* files. The images are copies of items already |
| 164 |
under git control. Also, wiki parsing can change the contents of Doc.* files frequently. |
| 165 |
|
| 166 |
Efforts have been made to centralize the definitions of wiki text markup to a single file should |
| 167 |
the decision be made to change wiki engines. This is a future proofing function. |