mirror of
https://github.com/valkey-io/valkey.git
synced 2026-05-08 22:39:38 -04:00
uriyage
e843c18f1e
## Add Fuzzing Capability to Valkey ### Overview This PR adds a fuzzing capability to Valkey, allowing developers and users to stress test their Valkey deployments with randomly generated commands. The fuzzer is integrated with the existing valkey-benchmark tool, making it easy to use without requiring additional dependencies. ### Key Features • **Command Generator**: Automatically generates Valkey commands by retrieving command information directly from the server • **Two Fuzzing Modes**: - normal: Generates only valid commands, doesn't modify server configurations - aggressive: Includes malformed commands and allows CONFIG SET operations • **Multi-threaded Testing**: Each client runs in a dedicated thread to maximize interaction between clients and enable testing of complicated scenarios • **Integration with valkey-benchmark**: Uses the existing CLI interface ### Implementation Details • Added new files: - `fuzzer_command_generator.h/c`: Dynamically generates valkey commands. - `fuzzer_client.c`: Orchestrate all the client threads, report test progress, and handle errors. • Modified existing files: - valkey-benchmark.c: Added fuzzing mode options and integration ### Command Generation Approach The fuzzer dynamically retrieves command information from the server, allowing it to adapt to different Valkey versions and custom modules. Since the command information generated from JSON files is sometimes limited, not all generated commands will be valid, but approximately 95% valid command generation is achieved. It is important to generate valid commands to cover as much code path as possible and not just the invalid command/args path. The fuzzer prioritizes generating syntactically and semantically correct commands to ensure thorough testing of the server's core functionality, while still including a small percentage of invalid commands in `aggressive` mode to test error handling paths #### Config modification For CONFIG SET command, the situation is more complex as the server currently provides limited information through CONFIG GET *. Some hardcoded logic is implemented that will need to be modified in the future. Ideally, the server should provide self-inspection commands to retrieve config keys-values with their properties (enum values, modifiability status, etc.). ### Issue Detection The fuzzer is designed to identify several types of issues: • Server crashes • Server memory corruptions / memory leaks(when compiled with ASAN) • Server unresponsiveness • Server malformed replies For unresponsiveness detection, command timeout limits are implemented to ensure no command blocks for excessive periods. If a server doesn't respond within 30 seconds, the fuzzer signals that something is wrong. ### Proven Effectiveness When running against the latest unstable version, the fuzzer has already identified several issues, demonstrating its effectiveness: * https://github.com/valkey-io/valkey/issues/2111 * https://github.com/valkey-io/valkey/issues/2112 * https://github.com/valkey-io/valkey/pull/2109 * https://github.com/valkey-io/valkey/pull/2113 * https://github.com/valkey-io/valkey/pull/2108 * https://github.com/valkey-io/valkey/pull/2137 * https://github.com/valkey-io/valkey/issues/2106 * https://github.com/valkey-io/valkey/pull/2347 * https://github.com/valkey-io/valkey/pull/2973 * https://github.com/valkey-io/valkey/pull/2974 ### How to Use Run the fuzzer using the valkey-benchmark tool with the --fuzz flag: ```bash # Basic usage (10000 commands 1000 commands per client, 10 clients) ./src/valkey-benchmark --fuzz -h 127.0.0.1 -p 6379 -n 10000 -c 10 # With aggressive fuzzing mode ./src/valkey-benchmark --fuzz --fuzz-level aggressive -h 127.0.0.1 -p 6379 -n 10000 -c 10 # With detailed logging ./src/valkey-benchmark --fuzz --fuzz-log-level debug -h 127.0.0.1 -p 6379 -n 10000 -c 10 ``` The fuzzer supports existing valkey-benchmark options, including TLS and cluster mode configuration. --------- Signed-off-by: Uri Yagelnik <uriy@amazon.com>
29 lines
710 B
C
29 lines
710 B
C
#ifndef FUZZER_COMMAND_GENERATOR_H
|
|
#define FUZZER_COMMAND_GENERATOR_H
|
|
|
|
#include <valkey/valkey.h>
|
|
#include "sds.h"
|
|
|
|
typedef struct FuzzerCommand {
|
|
sds *argv;
|
|
int argc;
|
|
int size;
|
|
} FuzzerCommand;
|
|
|
|
/* Fuzzing mode types */
|
|
typedef enum {
|
|
FUZZ_MODE_MALFORMED_COMMANDS = (1 << 0),
|
|
FUZZ_MODE_CONFIG_COMMANDS = (1 << 1)
|
|
} FuzzMode;
|
|
|
|
int initFuzzer(valkeyContext *ctx, int num_keys, int cluster_mode, int fuzz_flags);
|
|
void cleanupFuzzer(void);
|
|
void initThreadClientCtx(int fuzz_flags);
|
|
void resetClientFuzzCtx(void);
|
|
void freeClientCtx(void);
|
|
FuzzerCommand *generateCmd(void);
|
|
void freeCommand(FuzzerCommand *args);
|
|
char *printCommand(FuzzerCommand *cmd);
|
|
|
|
#endif /* FUZZER_COMMAND_GENERATOR_H */
|