Erlang Consistent Hashing

• Consistent_hashing
• Understanding and Practicing the Consistent Hashing Algorithm
• Massive Data Offload Processing - Consistent Hashing Algorithm
• What is the Consistent Hashing Algorithm

Introduction

Consistent_hashing is a special hashing algorithm. When using consistent hashing, changing the number of hash table slots (size) only requires remapping K/n keys on average, where K is the number of keys and n is the number of slots. However, in traditional hash tables, adding or deleting a slot requires remapping almost all keys.

Erlang Consistent Hashing

• hash_ring
• Usage

%% Build node information
NodeList = hash_ring:list_to_nodes([a,b,c,d,e]).
%% Generate hash ring data
HashRing = hash_ring:make(NodeList).
%% Map nodes based on key
hash_ring:find_node(xxx, HashRing).

• The generated consistent hash data must be saved for business purposes.
• You can use Erlang’s load module method, converting consistent hash data to xxx.erl and providing a corresponding fast data retrieval method to improve consistent hashing usage.

Extended Implementation

Code

• xh_hash
• Build

Usage

PoolName = account_server.
AccountServerNameList = [account_server_1, account_server_2, account_server_3, account_server_4, account_server_5].
xh_hash:make(PoolName, AccountServerNameList).
mhasxh_hashh:find_node(PoolName, 7010001100057200).
xh_hash:find_node(PoolName, 7010001100060800).
xh_hash:find_node(PoolName, 7010001100015500).
xh_hash:find_node(PoolName, 7010001100065900).
xh_hash:find_node(PoolName, 7010001100057000).

Custom Parameters

%% hash_algorithm Algorithm type [crc32, phash2, md5, sha, sha256]
%% virtual_node_count Number of virtual nodes
%% max_hash_byte_size Maximum hash byte size
OptionList = [
{hash_algorithm, phash2},
{virtual_node_count, 128},
{max_hash_byte_size, 8}
].
PoolName = account_server.
AccountServerNameList = [account_server_1, account_server_2, account_server_3, account_server_4, account_server_5].
xh_hash:make(PoolName, AccountServerNameList, OptionList).
xh_hash:find_node(PoolName, 7010001100057200).
xh_hash:find_node(PoolName, 7010001100060800).
xh_hash:find_node(PoolName, 7010001100015500).
xh_hash:find_node(PoolName, 7010001100065900).
xh_hash:find_node(PoolName, 7010001100057000).

Other

• Other hashing methods for expansion
• Unhandled dynamic node addition and deletion