build based on 4cc07b4

previews/PR521/advanced/index.html

@@ -43,4 +43,4 @@
 # to an `Upgrade` instance with the new struct.
 load(&quot;test.jld2&quot;, &quot;data&quot;; typemap=Dict(&quot;Main.OldStructVersion&quot; =&gt; JLD2.Upgrade(UpdatedStruct)))</code></pre><h2 id="Groups-Appending-to-files"><a class="docs-heading-anchor" href="#Groups-Appending-to-files">Groups - Appending to files</a><a id="Groups-Appending-to-files-1"></a><a class="docs-heading-anchor-permalink" href="#Groups-Appending-to-files" title="Permalink"></a></h2><p>Group objects can be constructed with two optional keyword arguments:</p><pre><code class="language-julia">g = Group(file;
           est_num_entries=4
-          est_link_name_len=8)</code></pre><p>These determine how much (additional) empty space should be allocated for the group description. (list of entries) This can be useful for performance when one expects to append many additional datasets after first writing the file.</p><h2 id="JLD2DebugTools"><a class="docs-heading-anchor" href="#JLD2DebugTools">JLD2DebugTools</a><a id="JLD2DebugTools-1"></a><a class="docs-heading-anchor-permalink" href="#JLD2DebugTools" title="Permalink"></a></h2><p>There is an experimental repository <a href="https://github.com/JonasIsensee/JLD2DebugTools.jl">JLD2DebugTools.jl</a> that may help with debugging files.</p><h2 id="Fallback-Behaviour"><a class="docs-heading-anchor" href="#Fallback-Behaviour">Fallback Behaviour</a><a id="Fallback-Behaviour-1"></a><a class="docs-heading-anchor-permalink" href="#Fallback-Behaviour" title="Permalink"></a></h2><p>By default JLD2 will attempt to open files using the <code>MmapIO</code> backend. If that fails, it retries using <code>IOStream</code>.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../hdf5compat/">« HDF5 Compatibility</a><a class="docs-footer-nextpage" href="../legacy/">Legacy »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Wednesday 3 January 2024 16:34">Wednesday 3 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+          est_link_name_len=8)</code></pre><p>These determine how much (additional) empty space should be allocated for the group description. (list of entries) This can be useful for performance when one expects to append many additional datasets after first writing the file.</p><h2 id="JLD2DebugTools"><a class="docs-heading-anchor" href="#JLD2DebugTools">JLD2DebugTools</a><a id="JLD2DebugTools-1"></a><a class="docs-heading-anchor-permalink" href="#JLD2DebugTools" title="Permalink"></a></h2><p>There is an experimental repository <a href="https://github.com/JonasIsensee/JLD2DebugTools.jl">JLD2DebugTools.jl</a> that may help with debugging files.</p><h2 id="Fallback-Behaviour"><a class="docs-heading-anchor" href="#Fallback-Behaviour">Fallback Behaviour</a><a id="Fallback-Behaviour-1"></a><a class="docs-heading-anchor-permalink" href="#Fallback-Behaviour" title="Permalink"></a></h2><p>By default JLD2 will attempt to open files using the <code>MmapIO</code> backend. If that fails, it retries using <code>IOStream</code>.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../hdf5compat/">« HDF5 Compatibility</a><a class="docs-footer-nextpage" href="../legacy/">Legacy »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Wednesday 3 January 2024 19:29">Wednesday 3 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

previews/PR521/compression/index.html

@@ -11,4 +11,4 @@
 
     # Don't compress this 
     write(f, "large_array", rand(10000))
-end</code></pre></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../customserialization/">« Custom Serialization</a><a class="docs-footer-nextpage" href="../internals/">Internals &amp; Design »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Wednesday 3 January 2024 16:34">Wednesday 3 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+end</code></pre></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../customserialization/">« Custom Serialization</a><a class="docs-footer-nextpage" href="../internals/">Internals &amp; Design »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Wednesday 3 January 2024 19:29">Wednesday 3 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

previews/PR521/customserialization/index.html

@@ -20,4 +20,4 @@
 
 arr = [B(rand()) for i=1:10]
 
-@save &quot;test.jld2&quot; arr</code></pre><p>In this example JLD2 converts the array of <code>B</code> structs to a plain <code>Vector{Float64}</code> prior to  storing to disk.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../">« Basics</a><a class="docs-footer-nextpage" href="../compression/">Compression »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Wednesday 3 January 2024 16:34">Wednesday 3 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+@save &quot;test.jld2&quot; arr</code></pre><p>In this example JLD2 converts the array of <code>B</code> structs to a plain <code>Vector{Float64}</code> prior to  storing to disk.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../">« Basics</a><a class="docs-footer-nextpage" href="../compression/">Compression »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Wednesday 3 January 2024 19:29">Wednesday 3 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

previews/PR521/hdf5compat/index.html

@@ -56,4 +56,4 @@
    }
 }
 }</code></pre><p>We can see that the file contains two things at top-level. There is a dataset <code>&quot;a&quot;</code> (that is what we wanted to store) and there is a group <code>_types</code> which is where all the necessary type information is stored.</p><p>You can see that JLD2 <em>committed</em> two compound datatypes. The first one is <code>Core.Datatype</code> which at first seems rather unintuitive. It is needed to tell HDF5 what a serialized  julia datatype looks like (a name and a list of parameters).</p><p>Below that is the definition of <code>MyCustomStruct</code> with two fields  <code>H5T_STD_I64LE &quot;x&quot;</code> and <code>H5T_IEEE_F64LE &quot;y&quot;</code> defining the integer field <code>x</code> and the float field <code>y</code>.</p><h2 id="A-note-on-pointers"><a class="docs-heading-anchor" href="#A-note-on-pointers">A note on pointers</a><a id="A-note-on-pointers-1"></a><a class="docs-heading-anchor-permalink" href="#A-note-on-pointers" title="Permalink"></a></h2><p>In the julia programming language pointers <code>Ptr</code> are not needed very often. However, when binary dependencies come into play and memory is passed back and forth, pointers do become relevant. Pointers are addresses to locations in memory and thus lose their meaning after a program has terminated.</p><p>In principle, there is little point in storing a pointer to a file but in order to allow for a more seamless experience JLD2 will, similar to <code>Base.Serialization</code> silently accept pointers. This is useful when storing large structures such as a <code>DifferentialEquations.jl</code> solution object that might contain a pointer somewhere. Upon deserialization any pointer fields are instantiated as null pointers.</p><p>This is done with just three lines of code utilizing the custom serialization logic and  it is shown here as it serves as a good example for usage of that feature.</p><pre><code class="language-julia">   writeas(::Type{&lt;:Ptr}) = Nothing
-   rconvert(::Type{Ptr{T}}, ::Nothing) where {T} = Ptr{T}()</code></pre><p>Usually one would also have to define a method for <code>wconvert</code>. However, in this  case JLD2 figures out that no explicit conversion is needed to construct <code>nothing</code>.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../internals/">« Internals &amp; Design</a><a class="docs-footer-nextpage" href="../advanced/">Advanced Usage »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Wednesday 3 January 2024 16:34">Wednesday 3 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+   rconvert(::Type{Ptr{T}}, ::Nothing) where {T} = Ptr{T}()</code></pre><p>Usually one would also have to define a method for <code>wconvert</code>. However, in this  case JLD2 figures out that no explicit conversion is needed to construct <code>nothing</code>.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../internals/">« Internals &amp; Design</a><a class="docs-footer-nextpage" href="../advanced/">Advanced Usage »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Wednesday 3 January 2024 19:29">Wednesday 3 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

previews/PR521/index.html

@@ -64,4 +64,4 @@
 julia> f["a"] # a new copy is loaded from the file
 2-element Vector{Float64}:
  0.0
- 0.0</code></pre><h3 id="Cross-compatibility"><a class="docs-heading-anchor" href="#Cross-compatibility">Cross-compatibility</a><a id="Cross-compatibility-1"></a><a class="docs-heading-anchor-permalink" href="#Cross-compatibility" title="Permalink"></a></h3><p>JLD2 tries to write files in a way that allows you to load them on different operating systems and in particular both on 32bit and 64bit systems. However, many julia structs may be inherently different on different architectures making this task impossible. In particular, moving data from a 64bit system to a 32bit system is only guaranteed to work for basic datatypes.</p><h3 id="Security"><a class="docs-heading-anchor" href="#Security">Security</a><a id="Security-1"></a><a class="docs-heading-anchor-permalink" href="#Security" title="Permalink"></a></h3><p>Beware of opening JLD2 files from untrusted sources. A malicious file may execute code on your computer. See e.g. <a href="https://github.com/JuliaIO/JLD2.jl/issues/117">here</a>. To check a file, you can use <a href="https://github.com/JonasIsensee/JLD2DebugTools.jl">JLD2DebugTools.jl</a> to view what kinds of objects are stored.</p></article><nav class="docs-footer"><a class="docs-footer-nextpage" href="customserialization/">Custom Serialization »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Wednesday 3 January 2024 16:34">Wednesday 3 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+ 0.0</code></pre><h3 id="Cross-compatibility"><a class="docs-heading-anchor" href="#Cross-compatibility">Cross-compatibility</a><a id="Cross-compatibility-1"></a><a class="docs-heading-anchor-permalink" href="#Cross-compatibility" title="Permalink"></a></h3><p>JLD2 tries to write files in a way that allows you to load them on different operating systems and in particular both on 32bit and 64bit systems. However, many julia structs may be inherently different on different architectures making this task impossible. In particular, moving data from a 64bit system to a 32bit system is only guaranteed to work for basic datatypes.</p><h3 id="Security"><a class="docs-heading-anchor" href="#Security">Security</a><a id="Security-1"></a><a class="docs-heading-anchor-permalink" href="#Security" title="Permalink"></a></h3><p>Beware of opening JLD2 files from untrusted sources. A malicious file may execute code on your computer. See e.g. <a href="https://github.com/JuliaIO/JLD2.jl/issues/117">here</a>. To check a file, you can use <a href="https://github.com/JonasIsensee/JLD2DebugTools.jl">JLD2DebugTools.jl</a> to view what kinds of objects are stored.</p></article><nav class="docs-footer"><a class="docs-footer-nextpage" href="customserialization/">Custom Serialization »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Wednesday 3 January 2024 19:29">Wednesday 3 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>