1 <img src="/static/logo.png" height="200px"/>
2
3 _Multilevel logging for advanced programs._
4
5 1. [lognestmonster](#lognestmonster)
6 2. [Library Class Structure](#library-class-structure)
7     1. [Semantics](#semantics)
8 3. [Serialization Format](#serialization-format)
9     1. [Events](#events)
10     2. [Statements](#statements)
11         1. [Verbosity Level Enumeration](#verbosity-level-enumeration)
12     3. [Example](#example)
13 4. [Temporary Data Saving](#temporary-data-saving)
14 5. [Copyright](#copyright)
15
16 # lognestmonster
17
18 ## Library Class Structure
19 This is subject to future change over security concerns regarding pointers and memory allocation.
20 ```
21 class lognestmonster
22
23     enum VerbosityLevels {INFO, DEBUG, VERBOSE, VERYVERBOSE, WARNING, ERROR}
24
25     struct QueueConfig
26         char * out_dir // directory to output log files
27
28     virtual void * alloc(size_t size)         // implementation defaults to cstd malloc()
29     virtual void free(void * block)           // implementation defaults to cstd free()
30     virtual void * serialize(LogObject * obj) // implementation defaults to manual serialization of standard LogObject to allocated block
31     virtual bool write(void * serialized, size_t size, std::ostream stream, bool append) // implementation defaults to writing entire serialized block to stream
32     virtual int delete(char * file_name)      // implementation defaults to cstd remove()
33
34     interface LogObject
35         Pushable * parent
36         char * temp_file
37         virtual bool save_temp()
38         virtual bool delete_temp()
39
40     interface Pushable
41     protected:
42         std::vector<LogObject *> pushed
43     public:
44         push(LogObject * obj)
45         push(int verbosity, char * tag, char * message) // implicitly creates a Statement and then pushes
46
47     class Queue : Pushable
48     public:
49         struct QueueConfig * _config
50         constructor (struct QueueConfig * config)
51         write()                                          // serializes, writes, and clears pushed LogObjects
52         write(LogObject * obj)                           // implicit push(), then write()
53         write(int verbosity, char * tag, char * message) // implicit Statement creation, push(), then write()
54
55     class Event : LogObject, Pushable
56     public:
57         constructor (LogObject * obj)                           // implicit push()
58         constructor (int verbosity, char * tag, char * message) // implicit Statement creation, then push()
59
60     class Statement : LogObject
61     public:
62         int verbosity
63         int timestamp
64         std::string * tag
65         std::string * message
66         constructor (int verbosity, char * tag, char * message)
67 ```
68 ### Semantics
69 A `Queue` handles data serialization and file writing to the main logtree file. Queue writing refers to sending serialized logtree data to the outstream. Queue pushing refers to adding an Event or Statement to the queue for future writing.
70
71 An `Event` is a pushable list of statements or events, or the "nest". Event pushing refers to adding an Event or Statement to the parent's list.
72
73 A `Statement` is the data-containing log item with a timestamp, verbosity level, tag/invoker, and message.
74
75 In reference to data serialization, `parser` as used here is just a deserializer.
76
77 ## Serialization Format
78
79 By default the library serializes log tree information in a special format. This can be overriden by anybody that uses the library.
80
81 ### Metadata
82
83 All saved files should begin with an `unsigned char` version number and an `unsigned long long` millisecond timestamp.
84 ```
85 unsigned char version
86 unsigned long long timestamp
87 ```
88 Version is representative of both the type of logtree and the format. Versions `0-99` should be indicative of Queues; `100-199` of Statements; `200-255` of Events.
89
90 ### Events
91 Open event with `0x2` and close with `0x3`. Statements or more events can be written inbetween these tags.
92 ```
93 0x2 // open event
94     // more events or statements
95 0x3 // close event
96 ```
97
98 ### Statements
99 Open statement with `0x0` and close `0x1`.
100
101 1. 1 byte for an open statement tag
102 2. 1 byte for a predefined verbosity level enum
103 3. 4 bytes for an unsigned integer timestamp
104 4. 1 byte for the length of the tag string
105 5. 0-255 bytes for the tag string
106 6. 2 bytes for the length of the message string
107 7. 0-65535 bytes for the message string
108 8. 1 byte for a close statement tag
109
110 ```
111 0x0
112     unsigned long long timestamp
113     unsigned char verbosity
114     unsigned char tag_size
115     unsigned char[] tag
116     unsigned short message_size
117     unsigned char[] message
118 0x1
119 ```
120 A close statement tag is always needed in case the serializer method is overriden and provides extra data/metadata. If a close statement tag isn't written, a parser/deserializer won't be able to read a serialized logtree with extra data.
121
122 #### Verbosity Level Enumeration
123 The 6 verbosity level enums and their byte values are:
124 ```
125 INFO        = 0
126 DEBUG       = 1
127 VERBOSE     = 2
128 VERYVERBOSE = 3
129 WARNING     = 4
130 ERROR       = 5
131 ```
132
133 ### Example
134 1 statement inside one 1 event:
135 ```
136 1565561768719 // timestamp    8*
137 0x2     // open event         1
138 0x0     // open statement     1
139 1565561768752 // timestamp    8*
140 0       // verbosity          1
141 4       // tag_size           1
142 "INIT"  // tag                4*
143 5       // message_size       2
144 "HELLO" // message            5*
145 0x1     // close statement    1
146 0x3     // close event        1
147         //                    33 total bytes for this log tree
148 ```
149 With the sample log tree used here, the raw byte file totals 25 bytes. In use, a parser/deserializer could take this file and create output similar to the following:
150 ```
151 lognestmonster - log_folder/
152 Size: 33 bytes | Timestamp: 1565561768719
153 1 Statement | 1 Event | 0 Unsaved Data Trees
154 ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
155
156 v 1 ITEM
157   1565561768752 - INFO - INIT - HELLO
158
159
160
161 ::::::::::: press q to exit | arrow keys to move, expand, collapse :::::::::::
162 ```
163
164 ## Temporary Data Saving
165
166 By the nature of a push-write logging library, there's a chance that some created Statements and Events might not be pushed and written before the program's exit, whether it hangs, crashes, throws a runtime exception, is SIGKILLed, or anything else. Seeing as the point of logging is to find and diagnose errors with ease, it'd be frustrating to lose critical last-second information like this. The solution: save temporary serialized data for every creation or change to Statements or Events. Every logtree that ends in a Statement will have its own temporary data file; when a Statement is pushed to an Event, the Statement's file will be deleted and replaced into the greater Event file. See the following example for how data is separated into files:
167
168 ```
169 Queue queue;
170 Event event;
171 Statement state1;
172 Statement state2;
173
174 // Existing files:
175 // statement1.raw
176 // statement2.raw
177
178 event.push(state1)
179
180 // Existing files:
181 // event.raw
182 // statement2.raw
183
184 queue.push(event)
185
186 // Existing files:
187 // event.raw
188 // statement2.raw
189
190 event.push(state2)
191
192 // Existing files:
193 // event.raw (all log items now exist inside the event, in the queue)
194
195 queue.write()
196
197 // Existing files:
198 // log12345.raw (consists of 2 statements inside 1 event)
199 ```
200
201 In reality, file names will likely contain timestamps, hashes, UUIDs, or some other form of identifiable metadata.
202
203 ## Copyright
204
205 lognestmonster Copyright (c) 2019 Joshua 'joshuas3' Stockin under the [GNU General Public License v3](LICENSE).
206
207 The following should be present in each file.
208 ```
209 lognestmonster Copyright (c) 2019 Joshua 'joshuas3' Stockin
210 <https://github.com/JoshuaS3/lognestmonster/>.
211
212
213 This file is part of lognestmonster.
214
215 lognestmonster is free software: you can redistribute it and/or modify
216 it under the terms of the GNU General Public License as published by
217 the Free Software Foundation, either version 3 of the License, or
218 (at your option) any later version.
219
220 lognestmonster is distributed in the hope that it will be useful,
221 but WITHOUT ANY WARRANTY; without even the implied warranty of
222 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
223 GNU General Public License for more details.
224
225 You should have received a copy of the GNU General Public License
226 along with lognestmonster. If not, see <https://www.gnu.org/licenses/>.
227 ```
228
1	<img src="/static/logo.png" height="200px"/>
2
3	_Multilevel logging for advanced programs._
4
5	1. [lognestmonster](#lognestmonster)
6	2. [Library Class Structure](#library-class-structure)
7	1. [Semantics](#semantics)
8	3. [Serialization Format](#serialization-format)
9	1. [Events](#events)
10	2. [Statements](#statements)
11	1. [Verbosity Level Enumeration](#verbosity-level-enumeration)
12	3. [Example](#example)
13	4. [Temporary Data Saving](#temporary-data-saving)
14	5. [Copyright](#copyright)
15
16	# lognestmonster
17
18	## Library Class Structure
19	This is subject to future change over security concerns regarding pointers and memory allocation.
20	```
21	class lognestmonster
22
23	enum VerbosityLevels {INFO, DEBUG, VERBOSE, VERYVERBOSE, WARNING, ERROR}
24
25	struct QueueConfig
26	char * out_dir // directory to output log files
27
28	virtual void * alloc(size_t size) // implementation defaults to cstd malloc()
29	virtual void free(void * block) // implementation defaults to cstd free()
30	virtual void * serialize(LogObject * obj) // implementation defaults to manual serialization of standard LogObject to allocated block
31	virtual bool write(void * serialized, size_t size, std::ostream stream, bool append) // implementation defaults to writing entire serialized block to stream
32	virtual int delete(char * file_name) // implementation defaults to cstd remove()
33
34	interface LogObject
35	Pushable * parent
36	char * temp_file
37	virtual bool save_temp()
38	virtual bool delete_temp()
39
40	interface Pushable
41	protected:
42	std::vector<LogObject *> pushed
43	public:
44	push(LogObject * obj)
45	push(int verbosity, char * tag, char * message) // implicitly creates a Statement and then pushes
46
47	class Queue : Pushable
48	public:
49	struct QueueConfig * _config
50	constructor (struct QueueConfig * config)
51	write() // serializes, writes, and clears pushed LogObjects
52	write(LogObject * obj) // implicit push(), then write()
53	write(int verbosity, char * tag, char * message) // implicit Statement creation, push(), then write()
54
55	class Event : LogObject, Pushable
56	public:
57	constructor (LogObject * obj) // implicit push()
58	constructor (int verbosity, char * tag, char * message) // implicit Statement creation, then push()
59
60	class Statement : LogObject
61	public:
62	int verbosity
63	int timestamp
64	std::string * tag
65	std::string * message
66	constructor (int verbosity, char * tag, char * message)
67	```
68	### Semantics
69	A `Queue` handles data serialization and file writing to the main logtree file. Queue writing refers to sending serialized logtree data to the outstream. Queue pushing refers to adding an Event or Statement to the queue for future writing.
70
71	An `Event` is a pushable list of statements or events, or the "nest". Event pushing refers to adding an Event or Statement to the parent's list.
72
73	A `Statement` is the data-containing log item with a timestamp, verbosity level, tag/invoker, and message.
74
75	In reference to data serialization, `parser` as used here is just a deserializer.
76
77	## Serialization Format
78
79	By default the library serializes log tree information in a special format. This can be overriden by anybody that uses the library.
80
81	### Metadata
82
83	All saved files should begin with an `unsigned char` version number and an `unsigned long long` millisecond timestamp.
84	```
85	unsigned char version
86	unsigned long long timestamp
87	```
88	Version is representative of both the type of logtree and the format. Versions `0-99` should be indicative of Queues; `100-199` of Statements; `200-255` of Events.
89
90	### Events
91	Open event with `0x2` and close with `0x3`. Statements or more events can be written inbetween these tags.
92	```
93	0x2 // open event
94	// more events or statements
95	0x3 // close event
96	```
97
98	### Statements
99	Open statement with `0x0` and close `0x1`.
100
101	1. 1 byte for an open statement tag
102	2. 1 byte for a predefined verbosity level enum
103	3. 4 bytes for an unsigned integer timestamp
104	4. 1 byte for the length of the tag string
105	5. 0-255 bytes for the tag string
106	6. 2 bytes for the length of the message string
107	7. 0-65535 bytes for the message string
108	8. 1 byte for a close statement tag
109
110	```
111	0x0
112	unsigned long long timestamp
113	unsigned char verbosity
114	unsigned char tag_size
115	unsigned char[] tag
116	unsigned short message_size
117	unsigned char[] message
118	0x1
119	```
120	A close statement tag is always needed in case the serializer method is overriden and provides extra data/metadata. If a close statement tag isn't written, a parser/deserializer won't be able to read a serialized logtree with extra data.
121
122	#### Verbosity Level Enumeration
123	The 6 verbosity level enums and their byte values are:
124	```
125	INFO = 0
126	DEBUG = 1
127	VERBOSE = 2
128	VERYVERBOSE = 3
129	WARNING = 4
130	ERROR = 5
131	```
132
133	### Example
134	1 statement inside one 1 event:
135	```
136	1565561768719 // timestamp 8*
137	0x2 // open event 1
138	0x0 // open statement 1
139	1565561768752 // timestamp 8*
140	0 // verbosity 1
141	4 // tag_size 1
142	"INIT" // tag 4*
143	5 // message_size 2
144	"HELLO" // message 5*
145	0x1 // close statement 1
146	0x3 // close event 1
147	// 33 total bytes for this log tree
148	```
149	With the sample log tree used here, the raw byte file totals 25 bytes. In use, a parser/deserializer could take this file and create output similar to the following:
150	```
151	lognestmonster - log_folder/
152	Size: 33 bytes \| Timestamp: 1565561768719
153	1 Statement \| 1 Event \| 0 Unsaved Data Trees
154	::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
155
156	v 1 ITEM
157	1565561768752 - INFO - INIT - HELLO
158
159
160
161	::::::::::: press q to exit \| arrow keys to move, expand, collapse :::::::::::
162	```
163
164	## Temporary Data Saving
165
166	By the nature of a push-write logging library, there's a chance that some created Statements and Events might not be pushed and written before the program's exit, whether it hangs, crashes, throws a runtime exception, is SIGKILLed, or anything else. Seeing as the point of logging is to find and diagnose errors with ease, it'd be frustrating to lose critical last-second information like this. The solution: save temporary serialized data for every creation or change to Statements or Events. Every logtree that ends in a Statement will have its own temporary data file; when a Statement is pushed to an Event, the Statement's file will be deleted and replaced into the greater Event file. See the following example for how data is separated into files:
167
168	```
169	Queue queue;
170	Event event;
171	Statement state1;
172	Statement state2;
173
174	// Existing files:
175	// statement1.raw
176	// statement2.raw
177
178	event.push(state1)
179
180	// Existing files:
181	// event.raw
182	// statement2.raw
183
184	queue.push(event)
185
186	// Existing files:
187	// event.raw
188	// statement2.raw
189
190	event.push(state2)
191
192	// Existing files:
193	// event.raw (all log items now exist inside the event, in the queue)
194
195	queue.write()
196
197	// Existing files:
198	// log12345.raw (consists of 2 statements inside 1 event)
199	```
200
201	In reality, file names will likely contain timestamps, hashes, UUIDs, or some other form of identifiable metadata.
202
203	## Copyright
204
205	lognestmonster Copyright (c) 2019 Joshua 'joshuas3' Stockin under the [GNU General Public License v3](LICENSE).
206
207	The following should be present in each file.
208	```
209	lognestmonster Copyright (c) 2019 Joshua 'joshuas3' Stockin
210	<https://github.com/JoshuaS3/lognestmonster/>.
211
212
213	This file is part of lognestmonster.
214
215	lognestmonster is free software: you can redistribute it and/or modify
216	it under the terms of the GNU General Public License as published by
217	the Free Software Foundation, either version 3 of the License, or
218	(at your option) any later version.
219
220	lognestmonster is distributed in the hope that it will be useful,
221	but WITHOUT ANY WARRANTY; without even the implied warranty of
222	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
223	GNU General Public License for more details.
224
225	You should have received a copy of the GNU General Public License
226	along with lognestmonster. If not, see <https://www.gnu.org/licenses/>.
227	```
228