You're paying loan interest. The carrier charges a rate (varies by product, often in the 4 to 6 percent range historically) on the outstanding loan. Meanwhile the cash value compounds at the crediting rate. The net economic benefit of the cash-value loan compared to a straight withdrawal is the crediting-rate-minus-loan-rate spread, not the full crediting rate.
On loaned dollars specifically, the spread is often small or even slightly negative depending on the product. The compound-twice framing is misleading because it ignores the cost of the loan.
The honest comparison: cash-value-loan-funded purchase vs alternative capital sources. If the alternative was a HELOC at 8% or credit at 12%, the policy loan is attractive. If the alternative was investing the same money in equities at 10% expected return, the policy loan loses.