Telecom network operators worldwide open their networks to each other to enable their mutual customers to communicate across network boundaries. This practice, known as “Interconnect,” is being used among national and international operators for fixed, mobile, and Internet services.  Network operators cross-charge each other for the interconnect services they offer each other’s customers. It is done through invoicing/billing and settlements. 

Network operators collect and store detailed information in a record known as Call Detail Record (CDR) about every call ever attempted, whether completed or not. A typical CDR captures data such as calling and called party phone numbers, duration of the call, CDR unique ID, a timestamp for each activity, the ID of the equipment that handled the call, the result of the call, and so on.  Interconnect partners share CDRs for the purpose of “verifying” cross charges and settling balances. This verification process is cumbersome, inefficient, lengthy, costly, and error-prone. Missing CDRs and discrepancies in CDRs are very common problems. 

Due to the large number of network operators globally and the many-to-many relationships amongst them, the complexity and amount of cross charging and settlements data is exponential and very error-prone, causing Interconnect to be very expensive. Interconnect is considered one of the largest operating costs and ranges from 30% to 50%, according to some estimates.  For example, to settle roaming charges, mobile operators check the International Data Clearing Houses (DCH) to validate roaming charges and then cross-reference the data against charges information provided by Financial Clearing Houses (FCH). FCH are used for debt collection rates and clearing accuracy. Access to DCH and FCH data is slow and the data itself is error prone. Network operators spend substantial amounts of time and money to fix data errors and to reconcile their records and books.  

A Blockchain based solution could be used to simplify and expedite the Interconnect’s cross-charging and billing data-verification process. Network operators would store CDR data related to the Interconnect cross-charging and billing into the Blockchain in real time. All operators will have instantaneous access to the data without being able to modify it.  The Blockchain ledger becomes the single source of truth, which allows network operators to access and verify billing and cross-charging data in real-time. It eliminates the need for data clearing houses and makes the reconciliation and settlement process simple and error-free.  It also helps fighting and preventing fraud. 

Network operators dedicate resources just to negotiate Interonnect charges and clearing agreements with other network operators.  Once reached, agreements are scripted and governed by international regulations. A Blockchain smart contract can be used to execute these agreements. Certified charging data can also be stored on the Blockchain, which would eliminate the need for financial clearing houses.

The direct beneficiary of the Blockchain-based solution are network operators that collectively constitute a business network or a closed ecosystem whose members have formal legal relationships and share a certain level, but not absolute, trust. This makes a permissioned DLT, such as the Hyperledger, more suitable than a permission-less public Blockchain. This is due to the nature of the relationships amongst network operators. For example, not all network operators need, or should have, access to all CDR data on the ledger; only those who have Interonnect agreements should have access to their mutual data. Also, CDR data contains confidential customer information and should not be open to the public. A permissioned DLT allows for controlled access to information on the ledger.

From the network operators’ perspective, the proposed Blockchain solution is efficiency enhancing, i.e., intensive margin, whereas, from the data and financial and clearing-houses perspective, it is a substitute threat.  

With the exception to the human interaction aspect related to negotiations and dispute resolution, Interconnect falls in the digital space.  This makes the last mile problem minimum and manageable. Basically, once Interconnect charges and billing terms and conditions are reached, all data generation, storage on the Blockchain, and verification is done automatically by the network. The last mile problem arises in situations when CDR data is in doubt.  In general, telecom network equipment manufacturers certify CDR data. However, there is no guarantee that some network operators may manipulate the data to their advantage before writing it on the Blockchain.  The solution is to establish some sort of governance and third-party auditors to ensure data providence and veracity. Data and financials clearing houses can be used to perform CDR data audits.  

The above solution is simplistic and intentionally did not address key details related to Interconnect regulatory aspects, types of agreements, call scenarios, invoicing, settlement, and netting, and reconciliation processes. These details have a great deal of impact on the how the smart contracts are structured, executed, and disputed in case of disagreements.


Proposal:

Develop a PoC to demonstrate a simple use case that involves a minimum of two carriers to post/update a minimum set of call related data on a DLT for the sake of settling cross-charges in realtime.  The scope, details, call flows, and development roadmap are subject to discussion and consensus within the group.

12 Comments

  1. Vipin Bharathan

    Beautiful writeup!

    A couple of points

    Using any ledger shared by the participants for private data has some challenging properties

    "only those who have Interconnect agreements should have access to their mutual data" could imply (in the HL Fabric context)

    a. bilateral channels or  private data collections- bilateral channels could imply explosion of channels (maybe in the context of Telecoms it is not very crucial) but it is n**2 for n participants. What is the rough measure of n for telecoms?

    b. netting across channels for bilateral will mean some kind of interoperability

    c. real-time settling could be another challenge as this might not take full advantage of netting; a solution could be to use an IOU token or a digital coin, and periodically settle them outside the system after netting.  Unable to render Jira issues macro, execution error.  Is a link to the FabToken proposal.

    d. I would opt for a PoC where you need at least three carriers to explore privacy and netting challenges noted above.

    e. Pruning of spent(settled and aged out) on-chain data will need some kind of governance, since if allowed to grow monotonically will have consequences.

    f. Maybe a totally different approach in the form of L2 protocols to preserve privacy and speed and data obsolescence could be the answer, with the ledger providing the anchor. 

    1. Vipin Rathi

      Thanks Vipin for your feedback.  Here is my answer to your questions:

      a. bilateral channels or  private data collections- bilateral channels could imply explosion of channels (maybe in the context of Telecoms it is not very crucial) but it is n**2 for n participants. What is the rough measure of n for telecoms?

         I agree best use case for bilateral channels are Banking applications but it might be useful in Telecoms as well depend on use case. . In the example of a three carrier network, Carrier A would have two bilateral channels, one each with Carrier B and Carrier C . For ensuring transactional validity, the endorsement policy for all bilateral transactions is set to require both participating carrier in each bilateral channel to endorse before committing into the ledger.

      Additionally, all carriers participate in two multilateral channels: funding channel and netting channel.


      Regards

      Vipin Rathi

      1. Vipin Bharathan

        Hello VipinR,

        My only question was the order of "n". Implementation wise; I agree with what you say about Fabric can be done for the bilateral channels. If you have netting and funding channels that are multilateral, they will leak data to competitors. Unless we have some FHE based netting algorithm as well as the results triggering funding flows.

        VipinB

  2. Bilal Saleh

    Thanks for the feedback Vipin... great points.  Here is my feedback:

    Using any ledger shared by the participants for private data has some challenging properties

    "only those who have Interconnect agreements should have access to their mutual data" could imply (in the HL Fabric context)

    • bilateral channels or  private data collections- bilateral channels could imply explosion of channels (maybe in the context of Telecoms it is not very crucial) but it is n**2 for n participants. What is the rough measure of n for telecoms? 
      • I don’t have the technical depth to assess the complexity and scalability issues.  Is Hyperledger Fabric the only option? Would we have the same issues with Sawtooth, for example?
    • netting across channels for bilateral will mean some kind of interoperability
      • Absolutely.  We have to address interoperability for syntax and semantics 
    • real-time settling could be another challenge as this might not take full advantage of netting; a solution could be to use an IOU token or a digital coin, and periodically settle them outside the system after netting.  
      • Using an internal token is a great idea for phase two or three.  To keep the PoC simple for phase one, I would focus on plumbing and flows, data format (interoperability), and a simple smart contract.
    • I would opt for a PoC where you need at least three carriers to explore privacy and netting challenges noted above.
      • Would be great if we could get three carriers to participate in the PoC
    • Pruning of spent (settled and aged out) on-chain data will need some kind of governance, since if allowed to grow monotonically will have consequences.
      • A simple governance model should be baked in the smart contract code.
    • Maybe a totally different approach in the form of L2 protocols to preserve privacy and speed and data obsolescence could be the answer, with the ledger providing the anchor. 
      • Not sure if I understand the point here.

    Thanks,

    Bilal

    1. Vipin Bharathan

      Hello Bilal,

      I agree that the PoC has to be kept simple. 

      Documenting the future path or what we do not address in PoC will always be useful, including our plans for handling some of the more thorny issues  towards a future MvP etc. so that we can anticipate questions comments that a PoC is bound to generate.

      Especially comments like "you have built a toy use case it will never work in the real world". I can go into depth on any of the points including Layer 2, when we have a more concrete proposal.

      Best,

      Vipin

  3. Mehul Shah

    Hello Bilal,

    Thanks for the excellent write-up.

    At the moment my sentiments are the blockchain as a system of record  is untenable for high-throughput use-cases.

    That being said, circumventing back to using blockchain technology to arbitrate the truth of course can't be discounted.

    My first question is what would the endorsement policy of a CDR being committed to the blockchain look like if for a moment we assume your suggestion of committing every CDR record to the blockchain is the architected solution.

    My second question is what does the current reconciliation and settlement process look like and settled from the arbitrator's perspective.

    Given some clarification, I want to table some architecture ideas that would impose a significantly low TPS load on the blockchain deployment using a combination of rolling hashes and summary only records persisted to the blockchain instead of every CDR record.

    I concur with Vipin, a PoC's design should be heavily influenced by forethought of a viable path from PoC to MVP.

    Best,
    Mehul

     

    1. Bilal Saleh

      Mehul,

      We need to delve into the use case details and come up with all call flow scenarios before attempting to define the architecture.  

      Particular issues we need to address include:

      • Carriers agreements types: bilateral, multilateral, unilateral 
      • Interconnection agreements
      • Interconnect call scenarios
      • Interconnect invoicing
      • Settlement process
      • Netting process
      • Reconciliation process

      For this exercise to move forward in a meaningful way, it is critical to have carriers and OSS/BSS/Billing vendors participation.


      Regards,

      Bilal

      1. Mehul Shah

        Bilal, agree we should delve into use case details before defining the architecture.

        When I broached the subject of architecture, it was confined to the tenability of ingesting CDR records into the blockchain considering the high volume and velocity at which CDR data is generated.

        That being said, would hate to be in a all-or-nothing situation and dismiss a whole host of use cases off hand as not possible given scalability constrains.

        My question is how can we bring the benefits of blockchain to the telecom sector for use cases contingent on CDR data. Is it prudent to entertain an architecture where CDR data is off-chain (sure we have to think about the can of worms this opens) but the excellent use cases you allude to are on chain.

  4. Vipin Bharathan

    When doing a PoC we need to look at load requirements for CDRs i.e. TPS and how it changes over time(week, month, season other patterns), size of payload etc. to make sure that we can handle it: Are there publicly available numbers for this?

  5. Vipin Rathi

    Hi Bilal,

    Thanks for starting this thread.

    I think CDR is the most common use case for Blockchain in Telecom. So, in my opinion, to save some time and efforts first we should start with a case study of CDR on Blockchain, next Architecture, then POC and last MVP. 

    To start a case study I am sharing some links as a reference. 

    1. https://www.tmforum.org/wp-content/uploads/2018/03/T.-Blockchain-Unleashed.pdf
    2. https://www2.deloitte.com/content/dam/Deloitte/za/Documents/technology-media-telecommunications/za_TMT_Blockchain_TelCo.pdf
    3. https://i3forum.org/wp-content/uploads/2018/05/3_A_blockchain_use_case_Shahar.pdf
    4. https://www.panamaxil.com/blog/blockchain-chaining-the-block-of-telcos

      I think CDR data related to the Interconnect cross-charging and billing into the Blockchain in real time creates a scalability issue.

    1. Scalability (Number of nodes): In our case, it would be 3 to 10 and that would be scalable in Hyperledger.
    2. I think Hyperledger  support up to 20k TPS.
    3. Scalability (Number of Clients): In Hyperledger thousands of clients can be connected.

    Regards

    Vipin Rathi

  6. Bilal Saleh

    It’s clear that scalability is a challenge and needs to be addressed head-on regardless of which use case we pick.  To optimize our efforts, I suggest we organize our work around two parallel tracks; performance and use case selection/definition/development, and create a roadmap for each.

    For the performance track, we need to:

    • Define telecom-specific performance criteria and metrics for non-realtime, billing-related applications such as number portability, roaming, and inter-connect.
      • What constitutes a transaction
      • Average TPS 
      • Average number of participating nodes
      • Incremental resources demand for additional nodes
      • Average CDR size 
      • Storage requirements as a function of CDR size
    • Benchmark Hyperledger performance. I am sure this has been done by the performance group
    • Determine performance gap between current state and requirements 
    • Work the Hyperledger technical to assess if the gap can be bridged and expected implementation timeframe 

    For the use case track, we need to:

    • Reach a consensus on the top three use cases and priorities their development 
    • For each use case:
      • Clearly define the specific problem it solves, target participating parties, call flow scenarios, limitations, and future improvements
      • Define PoC scope and technical and logistical requirements
      • Make sure to secure at least three telecom operators participants 
      • Identify key committed developers and a project manager
      • Do the work! 

    The use case track is important to demonstrate value at a functional level with the assumption that we have a roadmap to address the performance issue.

  7. Vipin Bharathan

    See Chris Ferris' note on Performance of Fabric on his blog, gives some hard numbers. For bilateral channel numbers N*(N-1)/2 
    https://www.ibm.com/blogs/blockchain/2019/04/does-hyperledger-fabric-perform-at-scale/?