Sublayout caching disabled when sublayout is hidden by serverside code

Issue

I have a leftmenu that can be configured on the page-template to be hidden using a Sitecore field. When the editor sets a checkmark in the HideLeftMenu field the placeholder containing the sc:Sublayout control is hidden. This seems to result in that caching of the Sublayout is ignored.
The code looks like this

Markup

 <asp:PlaceHolder runat="server" ID="phLeftColumn">  
   <div class="col-sm-4 col-md-3">  
     <asp:ContentPlaceHolder runat="server" ID="cphPageContentLeft">  
       <dom:Sublayout runat="server" ID="submenu" Path="/layouts/LeftMenu.ascx" Cacheable="True" VaryByData="True" />  
     </asp:ContentPlaceHolder>  
   </div>  
 </asp:PlaceHolder>  

Code behind

 phLeftColumn.Visible = Sitecore.Context.Item["HideLeftMenu"] != "1";