Debian Codegraph @

doc/wiki/scripts/README.WIKIPARSE

raw
1 * ----------------------------------------------------------------------
2 * Copyright (c) 2014 Scott Furry
3 *
4 * This file is part of Freedroid
5 *
6 * Freedroid is free software; you can redistribute it and/or modify
7 * it under the terms of the GNU General Public License as published by
8 * the Free Software Foundation; either version 2 of the License, or
9 * (at your option) any later version.
10 *
11 * Freedroid is distributed in the hope that it will be useful,
12 * but WITHOUT ANY WARRANTY; without even the implied warranty of
13 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14 * GNU General Public License for more details.
15 *
16 * You should have received a copy of the GNU General Public License
17 * along with Freedroid; see the file COPYING. If not, write to the
18 * Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
19 * MA 02111-1307 USA
20 *
21 * ----------------------------------------------------------------------
22 Freedroid RPG - wiki parsing README
23 ----------
24 Rationale:
25 ----------
26 Information on the wiki pages can sometimes get out of date or become inconsistent
27 with the current release. Wiki page editing/updating can sometimes become a major
28 chore espically with major changes to the code base.
29
30 ------------
31 Description:
32 ------------
33 Wiki Parsing automates the process of parsing game source files and updating wiki
34 pages containing game playing information. The wiki pages produced as a result
35 include NPC/Character/Item information and Map of levels in game.
36
37 ----------
38 Execution:
39 ----------
40 cd <path to FDRPG root>
41 lua doc/wiki/scripts/wiki_parse.lua [-s] [-l] [-v | -vv]
42 switches:
43 s enable sandbox output - disables some wiki text conversions
44 - output can be pasted into wiki sandbox for testing
45 (see assumptions: OUTPUT NOT HUMAN READABLE below)
46 l produce a list of file links
47 - saved as FDRPG root/doc/wiki/wiki.d/output_images.txt
48 (see assumptions: PATH STRUCTURE - FILE NAMES below)
49 v verbose output - produce verbose output to stdout
50 - note large scrollback produced
51 - prints out basic information about parsing
52 vv double verbose output - produce extreme verbose output to stdout
53 - implies verbose output
54 - note HUGE scrollback produced
55 - prints out basic information about parsing
56 - prints out detailed module table contents
57 RECOMMEND: pipe this output to file for easier review
58
59 e.g. lua doc/wiki/scripts/wiki_parse.lua [-v | -vv] > [path/]filename
60 -- output path should be used to prevent writing garbage to FDRPG root
61 -- suggest doc/wiki/wiki.d for path as .gitignore is set to ignore *.txt files
62
63 NOTE:
64 Wiki Parsing is intended to be used by the FDRPG buildbot as part of build process.
65
66 INPUT
67 All scripts necessary for wiki parsing are located at FDRPG root/doc/wiki/scripts
68
69 OUTPUT
70 Wiki Parsing will save files to the following locations:
71 wiki pages for uploading FRRPG root/doc/wiki/wiki.d/Doc.*
72 images for wiki pages FRRPG root/doc/wiki/uploads
73
74 These locations are then to be rsync'd with the FDRPG wiki site.
75
76 LUA MODULE LOADING
77 wiki_parse.lua processes the following FDRPG data files using the given lua code modules
78 droid_archetypes.dat - modWP_Droid.lua
79 events.dat - modWP_Events.lua
80 item_specs.lua - modWP_Items.lua
81 levels.dat - modWP_Levels.lua
82 npc_specs.lua - modWP_NPC.lua
83 ReturnOfTux.droids - modWP_ROTD.lua
84 quests.dat - modWP_Quests
85
86 For each module, the sequence used is:
87 - 'require' module (with path - see Assumptions:PATH STRUCTURE below)
88 - call module's "ProcessData" function -> process source files into data structs
89 - call module's WikiWrite function -> convert data structures to wiki pages
90 ## NOTE: some code modules may not have a WikiWrite function - but the module is needed to parse information.
91 The produced data is used by other modules for wiki pages.
92 ## NOTE: process modWP_Levels and modWP_NPC last (in that order) due to dependencies on other modules
93
94 FDRPG LUA DATA FILES
95 (i.e. item_specs.lua and npc_specs.lua )
96 FDRPG data files in lua format are parsed differently than plain text files.
97
98 Functions used to extract data must be in the lua global space or parsing will fail.
99 Also modWPCommon.Process.FileToChunk() function is used to removed gettext markers before processing as
100 lua's dofile() will fail if gettext markers are present.
101
102 Once the lua data files is loaded into memory as a "chunk", variables and structures in the data
103 file can be accessed directly by the wiki parsing script.
104
105 ------------
106 Assumptions:
107 ------------
108 BUILDBOT
109 Files from previous iterations/executions are removed by the buildbot.
110
111 PLATFORM
112 It is assumed that wiki_parsing.lua and its modules are being executed on a Linux-based platform.
113 Also, lua 5.2 is required. No other third-party lua modules/dependencies are used.
114
115 The script also executes a bash script for some file handling processes.
116
117 PAGE HISTORY
118 History of pmwiki page edits on auto-generated pages will not be retained between builds. The process
119 to keep this history is specific to pmwiki and not practical to integrate or emulate. A warning
120 is posted on auto-generated pages to users about the lack of edit/history retension.
121
122 Changes to the layout of auto-generated wiki pages should go through the Review Board process.
123
124 PATH STRUCTURE - FILE NAMES
125 Lua is not very aware of path structures and file names. The paths and file names used to find
126 lua modules and FDRPG data files had to be hardcoded into the wiki parsing source. Any changes made to
127 the overall FDRPG project structure will need to be pushed into wiki parsing.
128
129 When executing wiki_parsing, it is assumed that the current working directory is set to the FDRPG root.
130 Any other location will cause the script to fail and exit.
131
132 DATA FILES - TEXT MARKERS
133 Data files make frequent use of text markers (e.g. levels.dat "Levelnumber:") to identify data.
134 Assumption is that these markers do not change. If the markers are changed, changes must be
135 pushed into wiki parsing source.
136
137 OUTPUT NOT HUMAN READABLE
138 The output of wiki parsing is not meant to be human readable. The data written to file is intended to
139 be used directly as a wiki page. If one wishes to view the data, perform the following (current for pmwiki):
140 1) pass the flag "-s" to wiki_parse.lua
141 - this will set the value of modWPCommon.sandbox to "true" and disable some wiki conversions
142 2) open resulting doc/wiki/wiki.d/Doc.* file
143 3) copy everything after "text=" and before "time="
144 4) open the page https://www.freedroid.org/pmwiki/pmwiki.php?n=Main.WikiSandbox?action=edit in your browser
145 - FDRPG pmwiki sandbox page in edit mode
146 5) paste into the sandbox page text box
147 6) select "preview" button to view resulting wiki page
148
149 ----------
150 Features:
151 ----------
152 modWPCommon.lua function "WikifyText" replaces characters pmwiki considers "special".
153 see - http://www.pmwiki.org/wiki/PmWiki/PageFileFormat "text=" description.
154 Currently -> replace "%" with %25 <- must do this FIRST
155 replace "<" with %3c
156 replace "\n" with %0a
157
158 modWPCommon.lua function "extractText" - built to remove the gettext markers and quotation marks
159 from a string. This function is used frequently when extracting text data from FDRPG data files.
160 Adding/Removing gettext markers to data in source files should not cause the wiki parse process to
161 fail or produce bad results.
162
163 .gitignore files are used to exclude images and Doc.* files. The images are copies of items already
164 under git control. Also, wiki parsing can change the contents of Doc.* files frequently.
165
166 Efforts have been made to centralize the definitions of wiki text markup to a single file should
167 the decision be made to change wiki engines. This is a future proofing function.