hiredis-vip: Support redis cluster. Maintained and used at vipshop.

原作者: [db:作者] 来自: 网络收藏邀请

开源软件名称：

hiredis-vip

开源软件地址：

https://gitee.com/vipshop/hiredis-vip

开源软件介绍：

HIREDIS-VIP

Hiredis-vip is a C client library for the Redis database.

Hiredis-vip supported redis cluster.

Hiredis-vip fully contained and based on Hiredis .

CLUSTER SUPPORT

FEATURES:

SUPPORT REDIS CLUSTER:
- Connect to redis cluster and run commands.
SUPPORT MULTI-KEY COMMAND:
- Support MSET, MGET and DEL.
SUPPORT PIPELING:
- Support redis pipeline and can contain multi-key command like above.
SUPPORT Asynchronous API:
- User can run commands with asynchronous mode.

CLUSTER API:

redisClusterContext *redisClusterContextInit(void);void redisClusterFree(redisClusterContext *cc);int redisClusterSetOptionAddNode(redisClusterContext *cc, const char *addr);int redisClusterSetOptionAddNodes(redisClusterContext *cc, const char *addrs);int redisClusterSetOptionConnectBlock(redisClusterContext *cc);int redisClusterSetOptionConnectNonBlock(redisClusterContext *cc);int redisClusterSetOptionParseSlaves(redisClusterContext *cc);int redisClusterSetOptionParseOpenSlots(redisClusterContext *cc);int redisClusterSetOptionRouteUseSlots(redisClusterContext *cc);int redisClusterSetOptionConnectTimeout(redisClusterContext *cc, const struct timeval tv);int redisClusterSetOptionTimeout(redisClusterContext *cc, const struct timeval tv);int redisClusterSetOptionMaxRedirect(redisClusterContext *cc,  int max_redirect_count);int redisClusterConnect2(redisClusterContext *cc);void *redisClusterFormattedCommand(redisClusterContext *cc, char *cmd, int len);void *redisClustervCommand(redisClusterContext *cc, const char *format, va_list ap);void *redisClusterCommand(redisClusterContext *cc, const char *format, ...);void *redisClusterCommandArgv(redisClusterContext *cc, int argc, const char **argv, const size_t *argvlen);int redisClusterAppendFormattedCommand(redisClusterContext *cc, char *cmd, int len);int redisClustervAppendCommand(redisClusterContext *cc, const char *format, va_list ap);int redisClusterAppendCommand(redisClusterContext *cc, const char *format, ...);int redisClusterAppendCommandArgv(redisClusterContext *cc, int argc, const char **argv, const size_t *argvlen);int redisClusterGetReply(redisClusterContext *cc, void **reply);void redisClusterReset(redisClusterContext *cc);redisContext *ctx_get_by_node(redisClusterContext *cc, struct cluster_node *node);redisClusterAsyncContext *redisClusterAsyncConnect(const char *addrs, int flags);int redisClusterAsyncSetConnectCallback(redisClusterAsyncContext *acc, redisConnectCallback *fn);int redisClusterAsyncSetDisconnectCallback(redisClusterAsyncContext *acc, redisDisconnectCallback *fn);int redisClusterAsyncFormattedCommand(redisClusterAsyncContext *acc, redisClusterCallbackFn *fn, void *privdata, char *cmd, int len);int redisClustervAsyncCommand(redisClusterAsyncContext *acc, redisClusterCallbackFn *fn, void *privdata, const char *format, va_list ap);int redisClusterAsyncCommand(redisClusterAsyncContext *acc, redisClusterCallbackFn *fn, void *privdata, const char *format, ...);int redisClusterAsyncCommandArgv(redisClusterAsyncContext *acc, redisClusterCallbackFn *fn, void *privdata, int argc, const char **argv, const size_t *argvlen);void redisClusterAsyncDisconnect(redisClusterAsyncContext *acc);void redisClusterAsyncFree(redisClusterAsyncContext *acc);redisAsyncContext *actx_get_by_node(redisClusterAsyncContext *acc, cluster_node *node);

CLUSTER API (old api, version <= 0.3.0):

redisClusterContext *redisClusterConnect(const char *addrs, int flags);redisClusterContext *redisClusterConnectWithTimeout(const char *addrs, const struct timeval tv, int flags);redisClusterContext *redisClusterConnectNonBlock(const char *addrs, int flags);void redisClusterFree(redisClusterContext *cc);void redisClusterSetMaxRedirect(redisClusterContext *cc, int max_redirect_count);void *redisClusterFormattedCommand(redisClusterContext *cc, char *cmd, int len);void *redisClustervCommand(redisClusterContext *cc, const char *format, va_list ap);void *redisClusterCommand(redisClusterContext *cc, const char *format, ...);void *redisClusterCommandArgv(redisClusterContext *cc, int argc, const char **argv, const size_t *argvlen);redisContext *ctx_get_by_node(struct cluster_node *node, const struct timeval *timeout, int flags);int redisClusterAppendFormattedCommand(redisClusterContext *cc, char *cmd, int len);int redisClustervAppendCommand(redisClusterContext *cc, const char *format, va_list ap);int redisClusterAppendCommand(redisClusterContext *cc, const char *format, ...);int redisClusterAppendCommandArgv(redisClusterContext *cc, int argc, const char **argv, const size_t *argvlen);int redisClusterGetReply(redisClusterContext *cc, void **reply);void redisClusterReset(redisClusterContext *cc);redisClusterAsyncContext *redisClusterAsyncConnect(const char *addrs, int flags);int redisClusterAsyncSetConnectCallback(redisClusterAsyncContext *acc, redisConnectCallback *fn);int redisClusterAsyncSetDisconnectCallback(redisClusterAsyncContext *acc, redisDisconnectCallback *fn);int redisClusterAsyncFormattedCommand(redisClusterAsyncContext *acc, redisClusterCallbackFn *fn, void *privdata, char *cmd, int len);int redisClustervAsyncCommand(redisClusterAsyncContext *acc, redisClusterCallbackFn *fn, void *privdata, const char *format, va_list ap);int redisClusterAsyncCommand(redisClusterAsyncContext *acc, redisClusterCallbackFn *fn, void *privdata, const char *format, ...);int redisClusterAsyncCommandArgv(redisClusterAsyncContext *acc, redisClusterCallbackFn *fn, void *privdata, int argc, const char **argv, const size_t *argvlen);void redisClusterAsyncDisconnect(redisClusterAsyncContext *acc);void redisClusterAsyncFree(redisClusterAsyncContext *acc);

Quick usage

If you want used but not read the follow, please reference the examples:https://github.com/vipshop/hiredis-vip/wiki

Cluster synchronous API

To consume the synchronous API, there are only a few function calls that need to be introduced:

redisClusterContext *redisClusterContextInit(void);int redisClusterSetOptionAddNodes(redisClusterContext *cc, const char *addrs);int redisClusterSetOptionMaxRedirect(redisClusterContext *cc, int max_redirect_count);int redisClusterSetOptionConnectTimeout(redisClusterContext *cc, const struct timeval tv);int redisClusterSetOptionTimeout(redisClusterContext *cc, const struct timeval tv);int redisClusterConnect2(redisClusterContext *cc);void *redisClusterCommand(redisClusterContext *cc, const char *format, ...);void redisClusterFree(redisClusterContext *cc);

Cluster connecting

The function redisClusterContextInit is used to create a so-called redisClusterContext.The function redisClusterSetOptionAddNodes is used to add the redis cluster address.The function redisClusterConnect2 is used to connect to the redis cluser.The context is where Hiredis-vip Cluster holds state for connections. The redisClusterContextstruct has an integer err field that is non-zero when the connection is inan error state. The field errstr will contain a string with a description ofthe error.After trying to connect to Redis using redisClusterContext you shouldcheck the err field to see if establishing the connection was successful:

redisClusterContext *cc = redisClusterContextInit();redisClusterSetOptionAddNodes(cc, "127.0.0.1:6379,127.0.0.1:6380");redisClusterConnect2(cc);if (cc != NULL && cc->err) {    printf("Error: %s\n", cc->errstr);    // handle error}

Cluster sending commands

The next that will be introduced is redisClusterCommand.This function takes a format similar to printf. In the simplest form,it is used like this:

reply = redisClusterCommand(clustercontext, "SET foo bar");

The specifier %s interpolates a string in the command, and uses strlen todetermine the length of the string:

reply = redisClusterCommand(clustercontext, "SET foo %s", value);

Internally, Hiredis-vip splits the command in different arguments and willconvert it to the protocol used to communicate with Redis.One or more spaces separates arguments, so you can use the specifiersanywhere in an argument:

reply = redisClusterCommand(clustercontext, "SET key:%s %s", myid, value);

Cluster multi-key commands

Hiredis-vip supports mget/mset/del multi-key commands.Those multi-key commands is highly effective.Millions of keys in one mget command just used several seconds.

Example:

reply = redisClusterCommand(clustercontext, "mget %s %s %s %s", key1, key2, key3, key4);

Cluster cleaning up

To disconnect and free the context the following function can be used:

void redisClusterFree(redisClusterContext *cc);

This function immediately closes the socket and then frees the allocations done increating the context.

Cluster pipelining

The function redisClusterGetReply is exported as part of the Hiredis API and can be usedwhen a reply is expected on the socket. To pipeline commands, the only things that needsto be done is filling up the output buffer. For this cause, two commands can be used thatare identical to the redisClusterCommand family, apart from not returning a reply:

int redisClusterAppendCommand(redisClusterContext *cc, const char *format, ...);int redisClusterAppendCommandArgv(redisClusterContext *cc, int argc, const char **argv);

After calling either function one or more times, redisClusterGetReply can be used to receive thesubsequent replies. The return value for this function is either REDIS_OK or REDIS_ERR, wherethe latter means an error occurred while reading a reply. Just as with the other commands,the err field in the context can be used to find out what the cause of this error is.

void redisClusterReset(redisClusterContext *cc);

Warning: You must call redisClusterReset function after one pipelining anyway.

The following examples shows a simple cluster pipeline:

redisReply *reply;redisClusterAppendCommand(clusterContext,"SET foo bar");redisClusterAppendCommand(clusterContext,"GET foo");redisClusterGetReply(clusterContext,&reply); // reply for SETfreeReplyObject(reply);redisClusterGetReply(clusterContext,&reply); // reply for GETfreeReplyObject(reply);redisClusterReset(clusterContext);

Cluster asynchronous API

Hiredis-vip comes with an cluster asynchronous API that works easily with any event library.Now we just support and test for libevent and redis ae, if you need for other event libraries,please contact with us, and we will support it quickly.

Connecting

The function redisAsyncConnect can be used to establish a non-blocking connection toRedis. It returns a pointer to the newly created redisAsyncContext struct. The err fieldshould be checked after creation to see if there were errors creating the connection.Because the connection that will be created is non-blocking, the kernel is not able toinstantly return if the specified host and port is able to accept a connection.

redisClusterAsyncContext *acc = redisClusterAsyncConnect("127.0.0.1:6379", HIRCLUSTER_FLAG_NULL);if (acc->err) {    printf("Error: %s\n", acc->errstr);    // handle error}

The cluster asynchronous context can hold a disconnect callback function that is called when theconnection is disconnected (either because of an error or per user request). This function shouldhave the following prototype:

void(const redisAsyncContext *c, int status);

On a disconnect, the status argument is set to REDIS_OK when disconnection was initiated by theuser, or REDIS_ERR when the disconnection was caused by an error. When it is REDIS_ERR, the errfield in the context can be accessed to find out the cause of the error.

You not need to reconnect in the disconnect callback, hiredis-vip will reconnect this connection itselfwhen commands come to this redis node.

Setting the disconnect callback can only be done once per context. For subsequent calls it willreturn REDIS_ERR. The function to set the disconnect callback has the following prototype:

int redisClusterAsyncSetDisconnectCallback(redisClusterAsyncContext *acc, redisDisconnectCallback *fn);

Sending commands and their callbacks

In an cluster asynchronous context, commands are automatically pipelined due to the nature of an event loop.Therefore, unlike the cluster synchronous API, there is only a single way to send commands.Because commands are sent to Redis cluster asynchronously, issuing a command requires a callback functionthat is called when the reply is received. Reply callbacks should have the following prototype:

void(redisClusterAsyncContext *acc, void *reply, void *privdata);

The privdata argument can be used to curry arbitrary data to the callback from the point wherethe command is initially queued for execution.

The functions that can be used to issue commands in an asynchronous context are:

int redisClusterAsyncCommand(  redisClusterAsyncContext *acc,   redisClusterCallbackFn *fn,   void *privdata, const char *format, ...);

This function work like their blocking counterparts. The return value is REDIS_OK when the commandwas successfully added to the output buffer and REDIS_ERR otherwise. Example: when the connectionis being disconnected per user-request, no new commands may be added to the output buffer and REDIS_ERR isreturned on calls to the redisClusterAsyncCommand family.

If the reply for a command with a NULL callback is read, it is immediately freed. When the callbackfor a command is non-NULL, the memory is freed immediately following the callback: the reply is onlyvalid for the duration of the callback.

All pending callbacks are called with a NULL reply when the context encountered an error.

Disconnecting

An cluster asynchronous connection can be terminated using:

void redisClusterAsyncDisconnect(redisClusterAsyncContext *acc);

When this function is called, the connection is not immediately terminated. Instead, newcommands are no longer accepted and the connection is only terminated when all pending commandshave been written to the socket, their respective replies have been read and their respectivecallbacks have been executed. After this, the disconnection callback is executed with theREDIS_OK status and the context object is freed.

Hooking it up to event library X

There are a few hooks that need to be set on the cluster context object after it is created.See the adapters/ directory for bindings to ae and libevent.