The common solution to this is to require your supplier to enter into a so-called "escrow agreement" and to deposit the source code with a third-party agent such as NCC or Iron Mountain, who will release it to you on the occurrence of certain specified trigger events, typically including insolvency.
But, if you have the bargaining power, you can also try and negotiate additional protections in the software supply agreement itself.
And the recent case of Filmflex Movies v Piksel Limited demonstrates that the Courts are prepared to enforce such additional protections.
The dispute concerned a Master Services Agreement (MSA) for the supply and maintenance of video on demand movie streaming software, which contained contractual clauses governing the release of source code to the customer (Filmflex), plus a subsequent escrow agreement entered into by the parties. The trigger events in the supply agreement and escrow agreement were different: the former included the supplier (Piksel) appointing a third party developer; and a simple request from Filmflex for the release of the code.
Both of these trigger events occurred, but Piksel argued they had no obligation to release the source code because the events were not covered by the later escrow agreement which superseded whatever had been agreed in the contract.
The judge disagreed. The trigger events in the escrow agreement were as well as, not instead of, those freely negotiated in the contract. Moreover, Filmflex was entitled to delivery (on a memory stick, CD or by email) of the source code by virtue of the fact that in this case the intellectual property in the development platform in question was partly owned by both Filmflex and Piksel.
So don't feel you have to rely on the standard Ts and Cs of an escrow agreement. Consider additional contractual protections you might require, safe in the knowledge that they are likely to be enforceable.