here as linear projections. . -

[rifattryo.ut11]

The latest self-supervised loss is: In the inner loop, only θ is optimized and therefore written as a parameter of ℓ; θ are the "hyperparameters" of this loss function. In the outer loop, θK, θ, θ are optimized together with θ and are just a hidden state and not a parameter. The figure illustrates this difference with code, where θK and θ are implemented as layer parameters similar to the K parameter in self-attention. In general, all possible choices of θK, θ, θ constitute a series of multi-view reconstruction tasks. The outer loop can be understood as selecting a specific task from this task group.



For simplicity, the researchers designed all views japan mobile number currently developed are already very efficient in terms of the number of floating-point operations (). However, its update rule: cannot be parallelized because it depends on - in two positions: the negative sign and ▽. In response, the researchers proposed - gradient descent to represent the batch size. = ▽(';) used in the study where ' = – (,) where represents the last time step of the previous - (or the first - ) so that gradients can be computed in parallel at a time.



. Dual form The parallelization described above is necessary but not enough for the efficiency of "actual running time" (-k). However, in reality, it is impossible to compute all for a single one. Instead, an outer product is required to compute them one by one. Worse, for each is this will produce a larger memory footprint and cost than a large one. To solve these two problems, the researchers observed that: We don't actually need to concretize, . . . , as long as we can calculate and output k, .