Custom Hyperledger Fabric Network with Minifabric

Minifabric is a great tool to deploy test or even production-grade Hyperledger Fabric (HLF) networks. Recently, I had to set up a multi-channel HLF network where some parties were members of multiple channels. While Minifabric has a good set of introductory videos and documentation, I had to overcome a few caveats during my endeavor. For example, the Minifabric join command adds all the nodes to the given channel; hence, I had to worry about the order of network set up. Also, I wanted to use a particular HLF version, Node.js as chaincode language, CouchDB as database, and set port numbers. Further, Minifabric has evolved a bit from the introductory videos where more simple commands are introduced recently. This is my attempt to provide a detailed set of steps to build a HLF network with Minifabric.

We’ll set up the following consortium with 2 application channels, 3 organizations, and 2 orderers. Organizations Org1 and Org2 are members of Channel1 while organizations Org2 and Org3 are members of Channel2. Each channel has a single orderer node. We’ll set up this network and deploy a simple chaincode already available with Minifabric. Because Org2 is common to both the channels and the join command adds all nodes of the network to a given channel, we have to add it as a new organization to each channel. Therefore, I will first create 2 channels each with either Org1 and Org3, and then add Org2 to both channels. This means we have to set up 3 networks and later combine them to form the following network. We’ll also install and approve the same chaincode (in practice, you may install different chaincodes on different channels) from Org2 on both channels to demonstrate how this works.

Process

Following is the summary of my workflow:

1. Set up 3 networks — network1- org1, network2 - org2, and network3- org3
2. Setup 2 channels — channel1 - network1 and channel2 - network3
3. Add org2 into 2 channels — channel1 and channel2
4. Join peers from org2 to 2 channels — channel1 and channel2
5. Install chaincode on org2 peers— Install the chaincode on 2 peers and then approve them within other members of 2 channels

Define Networks

First, we need to define the 3 networks. Each network needs to be in a different working directory to enable Minifabric to create and manage scripts, templates, and intermediate files for each network instance in a separate vars directory. Thus, we’ll use 3 directories named org1, org2, and org3 to organize our networks. A network is defined using the spec.yaml file. It’s highly recommended to download the template file from the Minifabric Github repo and edit it. Following is the content of my network definition files:

cat ./org1/spec.yaml
fabric:
  cas:
  - "ca1.org1.example.com"
  peers: 
  - "peer1.org1.example.com"
  - "peer2.org1.example.com"
  orderers:
  - "orderer1.example.com"
  settings:
    ca:
      FABRIC_LOGGING_SPEC: DEBUG
    peer:
      FABRIC_LOGGING_SPEC: DEBUG
    orderer:
      FABRIC_LOGGING_SPEC: DEBUG
  netname: "network1"
cat ./org2/spec.yaml
fabric:
  cas:
  - "ca1.org2.example.com"
  peers: 
  - "peer1.org2.example.com"
  - "peer2.org2.example.com"
  settings:
    ca:
      FABRIC_LOGGING_SPEC: DEBUG
    peer:
      FABRIC_LOGGING_SPEC: DEBUG
  netname: "network2"
cat ./org3/spec.yaml
fabric:
  cas:
  - "ca1.org3.example.com"
  peers: 
  - "peer1.org3.example.com"
  - "peer2.org3.example.com"
  orderers:
  - "orderer2.example.com"
  settings:
    ca:
      FABRIC_LOGGING_SPEC: DEBUG
    peer:
      FABRIC_LOGGING_SPEC: DEBUG
    orderer:
      FABRIC_LOGGING_SPEC: DEBUG
  netname: "network3"

Make sure to set 3 different network names, as we want to launch all 3 networks simultaneously. We don’t have an orderer for network2 as we’ll be connecting org2 peers to the 2 channels each with an orderer. Also, note that while HLF recommends orderers and peers to be from different organizations, Minifabric enforces this by preventing us from using the same network for orderers and peers. Note that Minifabric will use cryptogen to generate certificates for our nodes instead of the CA. Therefore, we need CA for only the organizations that have users, e.g., orderer organization doesn’t need a CA as it’ll not manage any user accounts other than admin.

Note that I’ll be using some command parameters explicitly to enhance readability through they may be already included in the execution context of each network. Also, coupling multiple commands can seed up as Minifabric seems to stop its container everytime minifab command finish executing. It’s assumed that minifab is in the path. If not, add it to path using PATH=$PATH:<path to minifabric shell script or batch file>/:. or copy the file to /usr/local/bin

Launch Network 1 & 2

Next, we’ll launch network1 and network2, create 2 channels named channel1 and channel2, and then install simple chaincode that comes with Minifabric. We’ll also use HLF version 2.2 LTS, Node.js as chaincode language, CouchDB for database, and specific set of port numbers. We’ll be using the netup command as follows, as we want to launch the network using a custom set up:

cd org1
minifab netup -e 7100 -o org1.example.com -i 2.2 -l node -s couchdb

Use minifab -h command to find out the meaning of the above options. For example, -o indicates that we are launching the network as org1.example.com. Next, let’s create the channel and add nodes using the following command:

minifab create,join -c channel1

Note that the channel name must start with a character and can only contain lower-case letters and numbers. Let’s deploy the Node.js version of simple chaincode to our channel, approve and commit it, and then initialize it using the following command:

minifab install,approve,commit -n simple -l node -v 1.0 -p '"init","a","200","b","300"'

Now that 1st network is running, let’s launch the 3rd network, set up channel2, and install the chaincode using the following commands:

cd ../org3
minifab netup -e 7300 -o org3.example.com -i 2.2 -l node -s couchdb
minifab create,join -c channel2
minifab install,approve,commit -n simple -l node -v 1.0 -p '"init","a","200","b","300"'

Launch Network 2

It’s time to launch our 2nd network that needs to join both the channels.

cd ../org2
minifab netup -e 7200 -o org2.example.com -i 2.2 -l node -s couchdb

Join Network2 to Channels

Now that network2 is running, we can add it to the 2 application channels. We need to do this one channel at a time using the details in the JoinRequest_org2-example-com.json file produced by Minifabric. We use the recently introduced orgjoin command to add new nodes. We’ll also generate the profile files as we need to them to import orderers to org2 and join its peer to the 2 channels.

cd ../org1
cp ../org2/vars/JoinRequest_org2-example-com.json ./vars/NewOrgJoinRequest.json
minifab orgjoin,profilegen
cd ../org3
cp ../org2/vars/JoinRequest_org2-example-com.json ./vars/NewOrgJoinRequest.json
minifab orgjoin,profilegen

Join Org2 Peers to Channels

It’s time to add org2 to 2 channels. We can add org2 peers to channels using the following commands where we do a node import (detailed defined in endpoints.yamlfile) and then join all nodes in the network to the channel:

cd ../org2
cp ../org1/vars/profiles/endpoints.yaml vars
minifab nodeimport,join -c channel1
cp ../org3/vars/profiles/endpoints.yaml vars
minifab nodeimport,join -c channel2

Install Chaincode on Org2

It’s time to install the chaincode on org2 nodes. While still on the org2 directory, run the following commands:

minifab install,approve -n simple -v 1.0 -p '"init","a","200","b","300"' -c channel1

Finally, because org2 just joined, the chaincode must be approved again such that org2 can also commit it. Use the following commands on both channels. While, in this example, we used the same chaincode on both channels, you can install different chaincodes on different channels.

cd ../org1
minifab approve,discover,commit
cd ../org3
minifab approve,discover,commit

If all went well, we should see the following set of containers (use the command docker ps — format “{{.Names}}” — you can also use this command after major steps to make sure each network is set up correctly):

network3
ca1.org3.example.com
orderer2.example.com
peer2.org3.example.com
peer1.org3.example.com
peer2.org3.example.com.couchdb
peer1.org3.example.com.couchdb
network2
ca1.org2.example.com
peer2.org2.example.com
peer1.org2.example.com
peer2.org2.example.com.couchdb
peer1.org2.example.com.couchdb
network1
ca1.org1.example.com
orderer1.example.com
peer2.org1.example.com
peer1.org1.example.com
peer2.org1.example.com.couchdb
peer1.org1.example.com.couchdb

If things don’t go well, you can try to cleanup and start the broken network. However, this depends on where things went wrong. While time consuming, easiest option is to cleanup everything and then restart (executing commands using a shell script can help). I know this isn’t good advice but with limited manipulations Minifabric support it’s what we can do. I won’t blame Minifabric as it try to keep things simple!