| 1 |
/* Copyright (c) 2015, 2026, Oracle and/or its affiliates. |
| 2 |
|
| 3 |
This program is free software; you can redistribute it and/or modify |
| 4 |
it under the terms of the GNU General Public License, version 2.0, |
| 5 |
as published by the Free Software Foundation. |
| 6 |
|
| 7 |
This program is designed to work with certain software (including |
| 8 |
but not limited to OpenSSL) that is licensed under separate terms, |
| 9 |
as designated in a particular file or component or in included license |
| 10 |
documentation. The authors of MySQL hereby grant you an additional |
| 11 |
permission to link the program and your derivative works with the |
| 12 |
separately licensed software that they have either included with |
| 13 |
the program or referenced in the documentation. |
| 14 |
|
| 15 |
This program is distributed in the hope that it will be useful, |
| 16 |
but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 17 |
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 18 |
GNU General Public License, version 2.0, for more details. |
| 19 |
|
| 20 |
You should have received a copy of the GNU General Public License |
| 21 |
along with this program; if not, write to the Free Software |
| 22 |
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */ |
| 23 |
|
| 24 |
Testing plugin services |
| 25 |
----------------------- |
| 26 |
|
| 27 |
Plugin services are usually APIs provided by the mysqld server. Theses APIs can be tested with |
| 28 |
unit test frameworks like "googletest" by developers. In QA it is preferable to test plugin |
| 29 |
services closer to reality with the help of plugins due to following reasons: |
| 30 |
|
| 31 |
- Integrating plugin, plugin services, clients and SQL layer for testing interaction between all. |
| 32 |
- Having examples for users close to reality (as described in user manual). |
| 33 |
- Using our (qa) test tools. |
| 34 |
|
| 35 |
Implementing a test plugin |
| 36 |
-------------------------- |
| 37 |
|
| 38 |
Depending on the specification of a plugin service, the test plugin must have a special plugin type |
| 39 |
or even tested with all plugin types. |
| 40 |
|
| 41 |
For the frame work to implement test plugins, the plugin type "daemon" has be chosen. It will be |
| 42 |
loaded (started) with the sql statement "INSTALL PLUGIN" and unloaded (stopped) by calling |
| 43 |
"UNINSTALL PLUGIN". There is no other communication between server and daemon unless it will be |
| 44 |
explicitly implemented, e.g. with the help of an API or service. INSTALL and UNINSTALL will be |
| 45 |
executed by a mtr test. The mtr test may of course contain other sql statements to test the |
| 46 |
interaction between plugin (service) and SQL layer. |
| 47 |
|
| 48 |
The plugin shall write its test results into one or more files. These files can then be written |
| 49 |
into the result file of the mtr test with the normal mtr commands like "cat_file". |
| 50 |
|
| 51 |
With the help of the log_message service the test plugin can easily write messages into |
| 52 |
"mysqld.1.err". These messages can be made visible in the result file or checked as follows: |
| 53 |
Load this log into a table with 2 columns (date, info). The column "date" is ignored as not |
| 54 |
deterministic. The column "info" will be selected by removing "\r" to be compatible to windows |
| 55 |
(The err log file on windows contains CR,LF as newline). Select the messages so they will be |
| 56 |
visible in the mtr result file. |
| 57 |
|
| 58 |
A perl snippet can be used for that purpose, too. |
| 59 |
|
| 60 |
In a plugin status and system (sql) variables can be specified. With status variables the |
| 61 |
state of the test execution can be reported to the mtr test calling the test plugin. That might |
| 62 |
be useful to e.g. give a ready message for a test case or the whole plugin. Especially when |
| 63 |
using threads in a test plugin such status variables are useful to synchronize execution of |
| 64 |
plugin and mtr test as both are running in parallel. System variables can be used to e.g. control |
| 65 |
test execution. But, that is very rudimentary as can be used to configure the plugin as its start. |
| 66 |
|
| 67 |
Test plugins |
| 68 |
------------ |
| 69 |
|
| 70 |
There exist test plugins of type "daemon", which can be taken as examples or frameworks and |
| 71 |
calling mtr tests: |
| 72 |
|
| 73 |
- "plugin/test_services" for the test plugin written in C++, |
| 74 |
- "mysql-test/suite/test_services" for the corresponding mtr tests. |
| 75 |
|
| 76 |
Test plugins: |
| 77 |
|
| 78 |
- "test_services.cc" |
| 79 |
|
| 80 |
showing how to implement a test plugin for simple plugin services like "LogPluginErr" |
| 81 |
used to write messages to the error log. For each of the services there is a function |
| 82 |
implemented containing the test cases. Instead of using the standard I/O of C++ it is much |
| 83 |
more comfortable to used "my_open", "my_write", etc. as it covers all platforms. |
| 84 |
"LogPluginErr" writes into "mysqld.1.err". The (selected) contents of these files |
| 85 |
can be seen in the result file of "test_services.test". "test_services.test" didn't create |
| 86 |
a thread. The effect is that the test (mysqld) is waiting until the plugin completes its work. |
| 87 |
A waiting condition to synchronize execution of plugin and mtr test is not necessary. |
| 88 |
|
| 89 |
- "test_service_threaded.cc" |
| 90 |
|
| 91 |
runs the same tests as "test_services.cc", but in a thread. This needs synchronization |
| 92 |
of plugin and mtr test as both are running in parallel. The synchronization will be done |
| 93 |
by waiting for the plugin (exactly the thread in the plugin) to be finished. If the |
| 94 |
thread is needing a very long time then the mtr test may run in a timeout. |
| 95 |
|
| 96 |
- "test_framework.cc" |
| 97 |
|
| 98 |
is a rudimentary implementation of a daemon plugin, which can be take to create a new one. |
| 99 |
Of course copying and modifying/extending one of the other plugins to create a new |
| 100 |
test plugin can be recommended depending on the need of threads. |
| 101 |
|
| 102 |
Using threads has the advantage of limiting test execution with a timeout. That may be interesting |
| 103 |
for long running plugins, testing complex APIs. Plugins needing a short run time may run without threads. |
| 104 |
|
| 105 |
To create the plugins and run the test do the following: |
| 106 |
- modify "CmakeLists.txt" if you want to add a new test plugin, |
| 107 |
- rebuild ALL or only make test_services by running make in the test_services directory, |
| 108 |
- in mysql-test: "./mtr --suite=test_services test_services" or |
| 109 |
"./mtr --suite=test_services test_services_threaded" |
| 110 |
|
| 111 |
If a test case in plugin "test_services" shall be switched off run the test as follows: |
| 112 |
|
| 113 |
./mtr --mysqld=--loose-test-services-with-log-message=0 t/test_services.test |
| 114 |
|
| 115 |
That shows a simple way of control test execution with system variables. |
| 116 |
|
| 117 |
Steps to create a new test plugin |
| 118 |
--------------------------------- |
| 119 |
|
| 120 |
The following is an example of steps to create a test plugin for a plugin service relying on |
| 121 |
the daemon type: |
| 122 |
|
| 123 |
- Create test_<plugin name>.cc file under mysql/plugin/test_plugins deciding if threading |
| 124 |
is needed or not. |
| 125 |
- Put the test cases for plugin service to be tested in the init function (preferred) and |
| 126 |
the clean up in the deinit function. |
| 127 |
- Add new plugin to plugin.def to make it available to mtr. |
| 128 |
- Create one or more MTR tests. |
| 129 |
- MTR test loads plugin and execute the test cases. |
| 130 |
- After uninstall of the plugin put the test results into the result file of the mtr test. |
| 131 |
- Create a reference result file. |
| 132 |
|