a /Sic@sRdZddlZddlZddlZddlZddlmZddlmZddlm Z ddl m Z ddl m Z ddl mZdd l mZdd l mZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddlm Z ddl!m"Z"ddl!m#Z#ddl!m$Z$ddl!m%Z%ddl!m&Z&ddl'm(Z(ddl'm)Z)ddl*m+Z+ddZ,dd Z-d!d"Z.e+d#Gd$d#d#ej/Z0e+d%gd&Gd'd(d(ej/Z1e+d%gd&Gd)d%d%ej/Z2e1jd*e2_d+d,Z3Gd-d.d.e4Z5e+d/gd&Gd0d/d/ej6e5d1Z7e78e"9d/e7e+d/gd&Gd2d3d3e7Z:Gd4d5d5e:e j;Zd8\Z?Z@ZAd9d:ZBd;d<ZCd=d>ZDd?d@ZEGdAdBdBZFeGe.getterr")r*r+r,r"r)r# _make_getter<sr-VariableSynchronizationc@s eZdZdZdZdZdZdZdS)r.aTIndicates when a distributed variable will be synced. * `AUTO`: Indicates that the synchronization will be determined by the current `DistributionStrategy` (eg. With `MirroredStrategy` this would be `ON_WRITE`). * `NONE`: Indicates that there will only be one copy of the variable, so there is no need to sync. * `ON_WRITE`: Indicates that the variable will be updated across devices every time it is written. * `ON_READ`: Indicates that the variable will be aggregated across devices when it is read (eg. when checkpointing or when evaluating an op that uses the variable). Example: >>> temp_grad=[tf.Variable([0.], trainable=False, ... synchronization=tf.VariableSynchronization.ON_READ, ... aggregation=tf.VariableAggregation.MEAN ... )] rN)__name__ __module__ __qualname____doc__AUTONONEZON_WRITEON_READr"r"r"r#r.Es VariableAggregation)v1c@s0eZdZdZdZdZdZdZddZdd Z d S) VariableAggregationV2a@Indicates how a distributed variable will be aggregated. `tf.distribute.Strategy` distributes a model by making multiple copies (called "replicas") acting data-parallel on different elements of the input batch. When performing some variable-update operation, say `var.assign_add(x)`, in a model, we need to resolve how to combine the different values for `x` computed in the different replicas. * `NONE`: This is the default, giving an error if you use a variable-update operation with multiple replicas. * `SUM`: Add the updates across replicas. * `MEAN`: Take the arithmetic mean ("average") of the updates across replicas. * `ONLY_FIRST_REPLICA`: This is for when every replica is performing the same update, but we only want to perform the update once. Used, e.g., for the global step counter. rr/r0r1cCs t|jSr&hashvalueselfr"r"r#__hash__xszVariableAggregationV2.__hash__cCs2||ur dSt|tr*t|jt|jkSdSdS)NTF) isinstancer9intr>r@otherr"r"r#__eq__{s  zVariableAggregationV2.__eq__N) r2r3r4r5r7SUMMEANONLY_FIRST_REPLICArArFr"r"r"r#r;asr;c@s(eZdZdZdZdZdZdZddZdS)r9rr/r0r1cCs t|jSr&r<r?r"r"r#rAszVariableAggregation.__hash__N) r2r3r4r7rGrHrIZONLY_FIRST_TOWERrAr"r"r"r#r9s zC* `ONLY_FIRST_TOWER`: Deprecated alias for `ONLY_FIRST_REPLICA`. cCs|durtj}n>t|ttfsNz t|}Wn"tyLtd||Yn0|dur^tj}n0z t|}Wn"tytd||Yn0|dur|tjk}|||fS)zEGiven user-provided variable properties, sets defaults and validates.Nz6Invalid variable aggregation mode: {} for variable: {}z:Invalid variable synchronization mode: {} for variable: {}) r9r7rBr; ValueErrorformatr.r6r8)synchronization aggregation trainablenamer"r"r#.validate_synchronization_aggregation_trainables4       rPcsxeZdZdZddddddddddddejejdfddZdddddddddejejdf ddZ e j fdd Z Z S) VariableMetaclassz@Metaclass to allow construction of tf.Variable to be overridden.NTcCsZdd}tjD]\}}t||}q|dur4tj}|||||||||| | | | | ||dS)6Call on Variable class. Useful to force the signature.c[stdi|SN)N)r$r'r"r"r#z5VariableMetaclass._variable_v1_call..N) initial_valuerN collectionsvalidate_shapecaching_devicerO variable_defdtypeexpected_shape import_scope constraint use_resourcerLrMshaperget_default_graph_variable_creator_stackr-r9r7)clsrVrNrWrXrYrOrZr[r\r]r^r_rLrMr`previous_getterr r,r"r"r#_variable_v1_calls, z#VariableMetaclass._variable_v1_callc CsTdd} tjD]\}}t|| } q| dur4tj} | ||||||||| | | | d S)rRc[stdi|SrS)r%)kwsr"r"r#rTrUz5VariableMetaclass._variable_v2_call..N) rVrNrXrYrOrZr[r]r^rLrMr`ra)rdrVrNrXrYrOrZr[r]r^rLrMr`rer r,r"r"r#_variable_v2_calls& z#VariableMetaclass._variable_v2_callcsJ|tur|j|i|S|tur0|j|i|Stt|j|i|SdSr&) VariableV1rfVariablerhsuperrQ__call__)rdargsr( __class__r"r#rls zVariableMetaclass.__call__)r2r3r4r5r.r6r9r7rfrhrfilter_tracebackrl __classcell__r"r"rnr#rQs@ * #rQrjc @seZdZdZeddddddddddddejejdf ddZ dd Z d d Z d d Z ddZ eddZeddZeddZdxddZeddddZeddZeddZdyd d!Zdzd"d#Zd{d$d%Zd|d&d'Zd}d(d)Zd~d*d+Zdd,d-Zdd.d/Zdd0d1Zdd2d3Z dd4d5Z!dd6d7Z"dd8d9Z#dd:d;Z$ddd?Z&edd@dAdBZ'eddCddDdEZ(e)ddFdGZ*e+dHdIZ,e+dJdKZ-dLdMZ.dNdOZ/dPdQZ0dRdSZ1dTZ2edUdVZ3edWdXZ4edYdZZ5ed[d\Z6ed]d^Z7ed_d`Z8edadbZ9edcddZ:dedfZ;dgdhZdmdnZ?dodpZ@eddqdrdsZAdtduZBGdvdwdwZCdS)rja See the [variable guide](https://tensorflow.org/guide/variable). A variable maintains shared, persistent state manipulated by a program. The `Variable()` constructor requires an initial value for the variable, which can be a `Tensor` of any type and shape. This initial value defines the type and shape of the variable. After construction, the type and shape of the variable are fixed. The value can be changed using one of the assign methods. >>> v = tf.Variable(1.) >>> v.assign(2.) >>> v.assign_add(0.5) The `shape` argument to `Variable`'s constructor allows you to construct a variable with a less defined shape than its `initial_value`: >>> v = tf.Variable(1., shape=tf.TensorShape(None)) >>> v.assign([[1.]]) dtype=float32, numpy=array([[1.]], ...)> Just like any `Tensor`, variables created with `Variable()` can be used as inputs to operations. Additionally, all the operators overloaded for the `Tensor` class are carried over to variables. >>> w = tf.Variable([[1.], [2.]]) >>> x = tf.constant([[3., 4.]]) >>> tf.matmul(w, x) >>> tf.sigmoid(w + x) When building a machine learning model it is often convenient to distinguish between variables holding trainable model parameters and other variables such as a `step` variable used to count training steps. To make this easier, the variable constructor supports a `trainable=` parameter. `tf.GradientTape` watches trainable variables by default: >>> with tf.GradientTape(persistent=True) as tape: ... trainable = tf.Variable(1.) ... non_trainable = tf.Variable(2., trainable=False) ... x1 = trainable * 2. ... x2 = non_trainable * 3. >>> tape.gradient(x1, trainable) >>> assert tape.gradient(x2, non_trainable) is None # Unwatched Variables are automatically tracked when assigned to attributes of types inheriting from `tf.Module`. >>> m = tf.Module() >>> m.v = tf.Variable([1.]) >>> m.trainable_variables (,) This tracking then allows saving variable values to [training checkpoints](https://www.tensorflow.org/guide/checkpoint), or to [SavedModels](https://www.tensorflow.org/guide/saved_model) which include serialized TensorFlow graphs. Variables are often captured and manipulated by `tf.function`s. This works the same way the un-decorated function would have: >>> v = tf.Variable(0.) >>> read_and_decrement = tf.function(lambda: v.assign_sub(0.1)) >>> read_and_decrement() >>> read_and_decrement() Variables created inside a `tf.function` must be owned outside the function and be created only once: >>> class M(tf.Module): ... @tf.function ... def __call__(self, x): ... if not hasattr(self, "v"): # Or set self.v to None in __init__ ... self.v = tf.Variable(x) ... return self.v * x >>> m = M() >>> m(2.) >>> m(3.) >>> m.v See the `tf.function` documentation for details. NzA variable's value can be manually cached by calling tf.Variable.read_value() under a tf.device scope. The caching_device argument does not work properly.rYTc CstdS)aCreates a new variable with value `initial_value`. Args: initial_value: A `Tensor`, or Python object convertible to a `Tensor`, which is the initial value for the Variable. The initial value must have a shape specified unless `validate_shape` is set to False. Can also be a callable with no argument that returns the initial value when called. In that case, `dtype` must be specified. (Note that initializer functions from init_ops.py must first be bound to a shape before being used here.) trainable: If `True`, GradientTapes automatically watch uses of this variable. Defaults to `True`, unless `synchronization` is set to `ON_READ`, in which case it defaults to `False`. validate_shape: If `False`, allows the variable to be initialized with a value of unknown shape. If `True`, the default, the shape of `initial_value` must be known. caching_device: Note: This argument is only valid when using a v1-style `Session`. Optional device string describing where the Variable should be cached for reading. Defaults to the Variable's device. If not `None`, caches on another device. Typical use is to cache on the device where the Ops using the Variable reside, to deduplicate copying through `Switch` and other conditional statements. name: Optional name for the variable. Defaults to `'Variable'` and gets uniquified automatically. variable_def: `VariableDef` protocol buffer. If not `None`, recreates the Variable object with its contents, referencing the variable's nodes in the graph, which must already exist. The graph is not changed. `variable_def` and the other arguments are mutually exclusive. dtype: If set, initial_value will be converted to the given type. If `None`, either the datatype will be kept (if `initial_value` is a Tensor), or `convert_to_tensor` will decide. import_scope: Optional `string`. Name scope to add to the `Variable.` Only used when initializing from protocol buffer. constraint: An optional projection function to be applied to the variable after being updated by an `Optimizer` (e.g. used to implement norm constraints or value constraints for layer weights). The function must take as input the unprojected Tensor representing the value of the variable and return the Tensor for the projected value (which must have the same shape). Constraints are not safe to use when doing asynchronous distributed training. synchronization: Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class `tf.VariableSynchronization`. By default the synchronization is set to `AUTO` and the current `DistributionStrategy` chooses when to synchronize. aggregation: Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class `tf.VariableAggregation`. shape: (optional) The shape of this variable. If None, the shape of `initial_value` will be used. When setting this argument to `tf.TensorShape(None)` (representing an unspecified shape), the variable can be assigned with values of different shapes. Raises: ValueError: If both `variable_def` and initial_value are specified. ValueError: If the initial value is not specified, or does not have a shape and `validate_shape` is `True`. Nr) r@rVrNrXrYrOrZr[r]r^rLrMr`r"r"r#__init__osJzVariable.__init__cCstdSr&rr?r"r"r#__repr__szVariable.__repr__cCstdS)Returns the last snapshot of this variable. You usually do not need to call this method as all ops that need the value of the variable call it automatically through a `convert_to_tensor()` call. Returns a `Tensor` which holds the value of the variable. You can not assign a new value to this tensor as it is not a reference to the variable. To avoid copies, if the consumer of the returned value is on the same device as the variable, this actually returns the live value of the variable, not a copy. Updates to the variable are seen by the consumer. If the consumer is on a different device it will get a copy of the variable. Returns: A `Tensor` containing the value of the variable. Nrr?r"r"r#r>szVariable.valuecCstdS)Returns the value of this variable, read in the current context. Can be different from value() if it's on another device, with control dependencies, etc. Returns: A `Tensor` containing the value of the variable. Nrr?r"r"r# read_values zVariable.read_valuecCstdSzxOverrides the shape for this variable. Args: shape: the `TensorShape` representing the overridden shape. Nrr@r`r"r"r# set_shapeszVariable.set_shapecCstdSr&rr?r"r"r#rNszVariable.trainablecCstdSr&rr?r"r"r#rLszVariable.synchronizationcCstdSr&rr?r"r"r#rMszVariable.aggregationcCstdS)In a session, computes and returns the value of this variable. This is not a graph construction method, it does not add ops to the graph. This convenience method requires a session where the graph containing this variable has been launched. If no session is passed, the default session is used. See `tf.compat.v1.Session` for more information on launching a graph and on sessions. ```python v = tf.Variable([1, 2]) init = tf.compat.v1.global_variables_initializer() with tf.compat.v1.Session() as sess: sess.run(init) # Usage passing the session explicitly. print(v.eval(sess)) # Usage with the default session. The 'with' block # above makes 'sess' the default session. print(v.eval()) ``` Args: session: The session to use to evaluate this variable. If none, the default session is used. Returns: A numpy `ndarray` with a copy of the value of this variable. Nrr@sessionr"r"r#evalsz Variable.evalz{Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.csHt,ttjfddWdS1s:0YdS)aReturns the value of the initialized variable. You should use this instead of the variable itself to initialize another variable with a value that depends on the value of this variable. ```python # Initialize 'v' with a random tensor. v = tf.Variable(tf.random.truncated_normal([10, 40])) # Use `initialized_value` to guarantee that `v` has been # initialized before its value is used to initialize `w`. # The random values are picked only once. w = tf.Variable(v.initialized_value() * 2.0) ``` Returns: A `Tensor` holding the value of this variable after its initializer has run. csjSr&)rVr"r?r"r#rT)rUz,Variable.initialized_value..N)r init_scoper condis_variable_initializedrvr?r"r?r#initialized_values    zVariable.initialized_valuecCstdS)LReturns the Tensor used as the initial value for the variable. Note that this is different from `initialized_value()` which runs the op that initializes the variable before returning its value. This method returns the tensor that is used by the op that initializes the variable. Returns: A `Tensor`. Nrr?r"r"r#rV+s zVariable.initial_valuecCstdS)Returns the constraint function associated with this variable. Returns: The constraint function that was passed to the variable constructor. Can be `None` if no constraint was passed. Nrr?r"r"r#r^9szVariable.constraintFcCstdS)a?Assigns a new value to the variable. This is essentially a shortcut for `assign(self, value)`. Args: value: A `Tensor`. The new value for this variable. use_locking: If `True`, use locking during the assignment. name: The name of the operation to be created read_value: if True, will return something which evaluates to the new value of the variable; if False will return the assign op. Returns: The updated variable. If `read_value` is false, instead returns None in Eager mode and the assign op in graph mode. Nr)r@r> use_lockingrOrvr"r"r#assignCszVariable.assigncCstdS)a?Adds a value to this variable. This is essentially a shortcut for `assign_add(self, delta)`. Args: delta: A `Tensor`. The value to add to this variable. use_locking: If `True`, use locking during the operation. name: The name of the operation to be created read_value: if True, will return something which evaluates to the new value of the variable; if False will return the assign op. Returns: The updated variable. If `read_value` is false, instead returns None in Eager mode and the assign op in graph mode. Nrr@deltarrOrvr"r"r# assign_addUszVariable.assign_addcCstdS)aLSubtracts a value from this variable. This is essentially a shortcut for `assign_sub(self, delta)`. Args: delta: A `Tensor`. The value to subtract from this variable. use_locking: If `True`, use locking during the operation. name: The name of the operation to be created read_value: if True, will return something which evaluates to the new value of the variable; if False will return the assign op. Returns: The updated variable. If `read_value` is false, instead returns None in Eager mode and the assign op in graph mode. Nrrr"r"r# assign_subgszVariable.assign_subcCstdS)ahSubtracts `tf.IndexedSlices` from this variable. Args: sparse_delta: `tf.IndexedSlices` to be subtracted from this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: The updated variable. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. Nrr@ sparse_deltarrOr"r"r# scatter_subyszVariable.scatter_subcCstdS)aZAdds `tf.IndexedSlices` to this variable. Args: sparse_delta: `tf.IndexedSlices` to be added to this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: The updated variable. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. Nrrr"r"r# scatter_addszVariable.scatter_addcCstdS)aUpdates this variable with the max of `tf.IndexedSlices` and itself. Args: sparse_delta: `tf.IndexedSlices` to use as an argument of max with this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: The updated variable. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. Nrrr"r"r# scatter_maxszVariable.scatter_maxcCstdS)aUpdates this variable with the min of `tf.IndexedSlices` and itself. Args: sparse_delta: `tf.IndexedSlices` to use as an argument of min with this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: The updated variable. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. Nrrr"r"r# scatter_minszVariable.scatter_mincCstdS)a^Multiply this variable by `tf.IndexedSlices`. Args: sparse_delta: `tf.IndexedSlices` to multiply this variable by. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: The updated variable. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. Nrrr"r"r# scatter_mulszVariable.scatter_mulcCstdS)aZDivide this variable by `tf.IndexedSlices`. Args: sparse_delta: `tf.IndexedSlices` to divide this variable by. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: The updated variable. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. Nrrr"r"r# scatter_divszVariable.scatter_divcCstdS)a`Assigns `tf.IndexedSlices` to this variable. Args: sparse_delta: `tf.IndexedSlices` to be assigned to this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: The updated variable. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. Nrrr"r"r#scatter_updateszVariable.scatter_updatecCstdS)aAssigns `tf.IndexedSlices` to this variable batch-wise. Analogous to `batch_gather`. This assumes that this variable and the sparse_delta IndexedSlices have a series of leading dimensions that are the same for all of them, and the updates are performed on the last dimension of indices. In other words, the dimensions should be the following: `num_prefix_dims = sparse_delta.indices.ndims - 1` `batch_dim = num_prefix_dims + 1` `sparse_delta.updates.shape = sparse_delta.indices.shape + var.shape[ batch_dim:]` where `sparse_delta.updates.shape[:num_prefix_dims]` `== sparse_delta.indices.shape[:num_prefix_dims]` `== var.shape[:num_prefix_dims]` And the operation performed can be expressed as: `var[i_1, ..., i_n, sparse_delta.indices[i_1, ..., i_n, j]] = sparse_delta.updates[ i_1, ..., i_n, j]` When sparse_delta.indices is a 1D tensor, this operation is equivalent to `scatter_update`. To avoid this operation one can looping over the first `ndims` of the variable and using `scatter_update` on the subtensors that result of slicing the first dimension. This is a valid option for `ndims = 1`, but less efficient than this implementation. Args: sparse_delta: `tf.IndexedSlices` to be assigned to this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: The updated variable. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. Nrrr"r"r#batch_scatter_updates,zVariable.batch_scatter_updatecCstdS)aFApplies sparse subtraction to individual values or slices in a Variable. Assuming the variable has rank `P` and `indices` is a `Tensor` of rank `Q`. `indices` must be integer tensor, containing indices into self. It must be shape `[d_0, ..., d_{Q-2}, K]` where `0 < K <= P`. The innermost dimension of `indices` (with length `K`) corresponds to indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th dimension of self. `updates` is `Tensor` of rank `Q-1+P-K` with shape: ``` [d_0, ..., d_{Q-2}, self.shape[K], ..., self.shape[P-1]]. ``` For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this: ```python v = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8]) indices = tf.constant([[4], [3], [1] ,[7]]) updates = tf.constant([9, 10, 11, 12]) v.scatter_nd_sub(indices, updates) print(v) ``` After the update `v` would look like this: [1, -9, 3, -6, -4, 6, 7, -4] See `tf.scatter_nd` for more details about how to make updates to slices. Args: indices: The indices to be used in the operation. updates: The values to be used in the operation. name: the name of the operation. Returns: The updated variable. Nrr@indicesupdatesrOr"r"r#scatter_nd_subs,zVariable.scatter_nd_subcCstdS)a?Applies sparse addition to individual values or slices in a Variable. The Variable has rank `P` and `indices` is a `Tensor` of rank `Q`. `indices` must be integer tensor, containing indices into self. It must be shape `[d_0, ..., d_{Q-2}, K]` where `0 < K <= P`. The innermost dimension of `indices` (with length `K`) corresponds to indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th dimension of self. `updates` is `Tensor` of rank `Q-1+P-K` with shape: ``` [d_0, ..., d_{Q-2}, self.shape[K], ..., self.shape[P-1]]. ``` For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this: ```python v = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8]) indices = tf.constant([[4], [3], [1] ,[7]]) updates = tf.constant([9, 10, 11, 12]) v.scatter_nd_add(indices, updates) print(v) ``` The resulting update to v would look like this: [1, 13, 3, 14, 14, 6, 7, 20] See `tf.scatter_nd` for more details about how to make updates to slices. Args: indices: The indices to be used in the operation. updates: The values to be used in the operation. name: the name of the operation. Returns: The updated variable. Nrrr"r"r#scatter_nd_addGs,zVariable.scatter_nd_addcCstdS)aCApplies sparse assignment to individual values or slices in a Variable. The Variable has rank `P` and `indices` is a `Tensor` of rank `Q`. `indices` must be integer tensor, containing indices into self. It must be shape `[d_0, ..., d_{Q-2}, K]` where `0 < K <= P`. The innermost dimension of `indices` (with length `K`) corresponds to indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th dimension of self. `updates` is `Tensor` of rank `Q-1+P-K` with shape: ``` [d_0, ..., d_{Q-2}, self.shape[K], ..., self.shape[P-1]]. ``` For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this: ```python v = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8]) indices = tf.constant([[4], [3], [1] ,[7]]) updates = tf.constant([9, 10, 11, 12]) v.scatter_nd_update(indices, updates) print(v) ``` The resulting update to v would look like this: [1, 11, 3, 10, 9, 6, 7, 12] See `tf.scatter_nd` for more details about how to make updates to slices. Args: indices: The indices to be used in the operation. updates: The values to be used in the operation. name: the name of the operation. Returns: The updated variable. Nrrr"r"r#scatter_nd_updateus,zVariable.scatter_nd_updatecCstdS)aGather slices from params axis axis according to indices. This function supports a subset of tf.gather, see tf.gather for details on usage. Args: indices: The index `Tensor`. Must be one of the following types: `int32`, `int64`. Must be in range `[0, params.shape[axis])`. name: A name for the operation (optional). Returns: A `Tensor`. Has the same type as `params`. NAttributeErrorr@rrOr"r"r# sparse_readszVariable.sparse_readcCstdS)aVGather slices from `params` into a Tensor with shape specified by `indices`. See tf.gather_nd for details. Args: indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. Index tensor. name: A name for the operation (optional). Returns: A `Tensor`. Has the same type as `params`. Nrrr"r"r# gather_nds zVariable.gather_ndPrefer Dataset.range instead.cCstdS)Increments this variable until it reaches `limit`. When that Op is run it tries to increment the variable by `1`. If incrementing the variable would bring it above `limit` then the Op raises the exception `OutOfRangeError`. If no error is raised, the Op outputs the value of the variable before the increment. This is essentially a shortcut for `count_up_to(self, limit)`. Args: limit: value at which incrementing the variable raises an error. Returns: A `Tensor` that will hold the variable value before the increment. If no other Op modifies this variable, the values produced will all be distinct. Nrr@limitr"r"r# count_up_toszVariable.count_up_tozr|r"r"r#loads#  z Variable.loadcCsN|}|r6||js6td|jd|jjd|d|rB|S|SdS)7Utility function for converting a Variable to a Tensor.z0Incompatible type conversion requested to type 'z' for variable of type 'z ' (Variable: z).Nis_compatible_withr[rJrO_refr>vr[rOas_refr r"r"r#_TensorConversionFunctions z"Variable._TensorConversionFunctioncCs*tjjD]}||qt|dtjdS)z%Register overloads for all operators. __getitem__N)rTensorOVERLOADABLE_OPERATORS_OverloadOperatorsetattrr _SliceHelperVar)rdoperatorr"r"r#_OverloadAllOperatorss  zVariable._OverloadAllOperatorscsH|dks|dkrdSttj|fdd}t|t|||dS)zDefer an operator overload to `ops.Tensor`. We pull the operator out of ops.Tensor dynamically to avoid ordering issues. Args: operator: string. The operator name. rF__ne__Ncs|g|Ri|Sr&)r>)armr(Z tensor_operr"r#_run_op0sz+Variable._OverloadOperator.._run_op)getattrrr functoolsupdate_wrapperr)rdrrr"rr#rs    zVariable._OverloadOperatorcCs.tjjr"tr"td|dnt|SdS)NzKVariable is unhashable. Instead, use variable.ref() as the key. (Variable: ))rr _USE_EQUALITY#executing_eagerly_outside_functions TypeErroridr?r"r"r#rA7szVariable.__hash__cCs,tjjr tr tj||ddS||uSdSz1Compares two variables element-wise for equality.F)incompatible_shape_errorN)rrrrr equalrDr"r"r#rF@szVariable.__eq__cCs,tjjr tr tj||ddS||uSdSr)rrrrr not_equalrDr"r"r#rIszVariable.__ne__cCs t|S)z@When executing eagerly, iterates over the value of the variable.)iterrvr?r"r"r#__iter__QszVariable.__iter__dcCstdS)The name of this variable.Nrr?r"r"r#rO^sz Variable.namecCs|jd|jdS)zThe shared name of the variable. Unlike name(), shared_name doesn't have ":0" suffix. It is user-specified name with name scope prefix. Returns: variable name. N:)rOindexr?r"r"r# _shared_namecs zVariable._shared_namecCstdS),The initializer operation for this variable.Nrr?r"r"r#roszVariable.initializercCstdS)The device of this variable.Nrr?r"r"r#devicetszVariable.devicecCstdS)The `DType` of this variable.Nrr?r"r"r#r[yszVariable.dtypecCstdS)!The `Operation` of this variable.Nrr?r"r"r#op~sz Variable.opcCstdS)The `Graph` of this variable.Nrr?r"r"r#graphszVariable.graphcCstdS)MThe `TensorShape` of this variable. Returns: A `TensorShape`. Nrr?r"r"r#r`szVariable.shapecCs|jS)zAlias of `Variable.shape`.)r`r?r"r"r# get_shapeszVariable.get_shapecCs tj|iS)zAFor implementing `Trackable`. This object is saveable on its own.) trackableVARIABLE_VALUE_KEYr?r"r"r# _gather_saveables_for_checkpointsz)Variable._gather_saveables_for_checkpointcCstdS)Converts a `Variable` to a `VariableDef` protocol buffer. Args: export_scope: Optional `string`. Name scope to remove. Returns: A `VariableDef` protocol buffer, or `None` if the `Variable` is not in the specified name scope. Nr)r@ export_scoper"r"r#to_protos zVariable.to_protocCs t||dS)z8Returns a `Variable` object created from `variable_def`.rZr]) RefVariablerr"r"r# from_protoszVariable.from_protocCs ||_dS)zrSets the slice info for this `Variable`. Args: save_slice_info: A `Variable.SaveSliceInfo` object. N_save_slice_info)r@save_slice_infor"r"r#_set_save_slice_infoszVariable._set_save_slice_infocCs|jSr&rr?r"r"r#_get_save_slice_infoszVariable._get_save_slice_infozUse ref() instead.cCs|Sr&)refr?r"r"r#experimental_refszVariable.experimental_refcCs t|S)aReturns a hashable reference object to this Variable. The primary use case for this API is to put variables in a set/dictionary. We can't put variables in a set/dictionary as `variable.__hash__()` is no longer available starting Tensorflow 2.0. The following will raise an exception starting 2.0 >>> x = tf.Variable(5) >>> y = tf.Variable(10) >>> z = tf.Variable(10) >>> variable_set = {x, y, z} Traceback (most recent call last): ... TypeError: Variable is unhashable. Instead, use tensor.ref() as the key. >>> variable_dict = {x: 'five', y: 'ten'} Traceback (most recent call last): ... TypeError: Variable is unhashable. Instead, use tensor.ref() as the key. Instead, we can use `variable.ref()`. >>> variable_set = {x.ref(), y.ref(), z.ref()} >>> x.ref() in variable_set True >>> variable_dict = {x.ref(): 'five', y.ref(): 'ten', z.ref(): 'ten'} >>> variable_dict[y.ref()] 'ten' Also, the reference object provides `.deref()` function that returns the original Variable. >>> x = tf.Variable(5) >>> x.ref().deref() )r Referencer?r"r"r#rs'z Variable.refc@s0eZdZdZd ddZeddZd ddZdS) zVariable.SaveSliceInfoa!Information on how to save this Variable as a slice. Provides internal support for saving variables as slices of a larger variable. This API is not public and is subject to change. Available properties: * full_name * full_shape * var_offset * var_shape NcCsz|r^t|tjsJtj|j|d|_dd|jD|_dd|jD|_dd|jD|_n||_||_||_||_dS)aCreate a `SaveSliceInfo`. Args: full_name: Name of the full variable of which this `Variable` is a slice. full_shape: Shape of the full variable, as a list of int. var_offset: Offset of this `Variable` into the full variable, as a list of int. var_shape: Shape of this `Variable`, as a list of int. save_slice_info_def: `SaveSliceInfoDef` protocol buffer. If not `None`, recreates the SaveSliceInfo object its contents. `save_slice_info_def` and other arguments are mutually exclusive. import_scope: Optional `string`. Name scope to add. Only used when initializing from protocol buffer. r]cSsg|]}|qSr"r".0ir"r"r# rUz3Variable.SaveSliceInfo.__init__..cSsg|]}|qSr"r"rr"r"r#r rUcSsg|]}|qSr"r"rr"r"r#r rUN) rBrSaveSliceInfoDefrprepend_name_scope full_name full_shape var_offset var_shape)r@rrrrsave_slice_info_defr]r"r"r#rrszVariable.SaveSliceInfo.__init__cCs@ddd|jDd}dddt|j|jD}||S)z)Computes the spec string used for saving. css|]}d|VqdS)z%dNr")rdr"r"r# rUz.Variable.SaveSliceInfo.spec..rcss|]\}}d||fVqdS)z%d,%dNr")rosr"r"r#rs)joinrziprr)r@full_shape_strZsl_specr"r"r#specs   zVariable.SaveSliceInfo.speccCs|dus|j|rxt}t|j||_|jD]}|j|q2|jD]}|j|qJ|j D]}|j |qb|SdSdS)zReturns a SaveSliceInfoDef() proto. Args: export_scope: Optional `string`. Name scope to remove. Returns: A `SaveSliceInfoDef` protocol buffer, or None if the `Variable` is not in the specified name scope. N) r startswithrrrstrip_name_scoperappendrr)r@rrrr"r"r#rs    zVariable.SaveSliceInfo.to_proto)NNNNNN)N)r2r3r4r5rrpropertyrrr"r"r"r# SaveSliceInfos # r)N)FNT)FNT)FNT)FN)FN)FN)FN)FN)FN)FN)FN)N)N)N)N)N)N)NNF)N)N)Dr2r3r4r5rr.r6r9r7rrrsr>rvryrrNrLrMr}rrrVr^rrrrrrrrrrrrrrrrrr staticmethodr classmethodrrrArFrr__array_priority__rOrrrr[rrr`rrrrrrrrrr"r"r"r#rjs] H                  . . . .    ,                 )) metaclassc@sBeZdZdZddddddddddddejejdfddZe j Z dS)riaSee the [Variables Guide](https://tensorflow.org/guide/variables). A variable maintains state in the graph across calls to `run()`. You add a variable to the graph by constructing an instance of the class `Variable`. The `Variable()` constructor requires an initial value for the variable, which can be a `Tensor` of any type and shape. The initial value defines the type and shape of the variable. After construction, the type and shape of the variable are fixed. The value can be changed using one of the assign methods. If you want to change the shape of a variable later you have to use an `assign` Op with `validate_shape=False`. Just like any `Tensor`, variables created with `Variable()` can be used as inputs for other Ops in the graph. Additionally, all the operators overloaded for the `Tensor` class are carried over to variables, so you can also add nodes to the graph by just doing arithmetic on variables. ```python import tensorflow as tf # Create a variable. w = tf.Variable(, name=) # Use the variable in the graph like any Tensor. y = tf.matmul(w, ...another variable or tensor...) # The overloaded operators are available too. z = tf.sigmoid(w + y) # Assign a new value to the variable with `assign()` or a related method. w.assign(w + 1.0) w.assign_add(1.0) ``` When you launch the graph, variables have to be explicitly initialized before you can run Ops that use their value. You can initialize a variable by running its *initializer op*, restoring the variable from a save file, or simply running an `assign` Op that assigns a value to the variable. In fact, the variable *initializer op* is just an `assign` Op that assigns the variable's initial value to the variable itself. ```python # Launch the graph in a session. with tf.compat.v1.Session() as sess: # Run the variable initializer. sess.run(w.initializer) # ...you now can run ops that use the value of 'w'... ``` The most common initialization pattern is to use the convenience function `global_variables_initializer()` to add an Op to the graph that initializes all the variables. You then run that Op after launching the graph. ```python # Add an Op to initialize global variables. init_op = tf.compat.v1.global_variables_initializer() # Launch the graph in a session. with tf.compat.v1.Session() as sess: # Run the Op that initializes global variables. sess.run(init_op) # ...you can now run any Op that uses variable values... ``` If you need to create a variable with an initial value dependent on another variable, use the other variable's `initialized_value()`. This ensures that variables are initialized in the right order. All variables are automatically collected in the graph where they are created. By default, the constructor adds the new variable to the graph collection `GraphKeys.GLOBAL_VARIABLES`. The convenience function `global_variables()` returns the contents of that collection. When building a machine learning model it is often convenient to distinguish between variables holding the trainable model parameters and other variables such as a `global step` variable used to count training steps. To make this easier, the variable constructor supports a `trainable=` parameter. If `True`, the new variable is also added to the graph collection `GraphKeys.TRAINABLE_VARIABLES`. The convenience function `trainable_variables()` returns the contents of this collection. The various `Optimizer` classes use this collection as the default list of variables to optimize. WARNING: tf.Variable objects by default have a non-intuitive memory model. A Variable is represented internally as a mutable Tensor which can non-deterministically alias other Tensors in a graph. The set of operations which consume a Variable and can lead to aliasing is undetermined and can change across TensorFlow versions. Avoid writing code which relies on the value of a Variable either changing or not changing as other operations happen. For example, using Variable objects or simple functions thereof as predicates in a `tf.cond` is dangerous and error-prone: ``` v = tf.Variable(True) tf.cond(v, lambda: v.assign(False), my_false_fn) # Note: this is broken. ``` Here, adding `use_resource=True` when constructing the variable will fix any nondeterminism issues: ``` v = tf.Variable(True, use_resource=True) tf.cond(v, lambda: v.assign(False), my_false_fn) ``` To use the replacement for variables which does not have these issues: * Add `use_resource=True` when constructing `tf.Variable`; * Call `tf.compat.v1.get_variable_scope().set_use_resource(True)` inside a `tf.compat.v1.variable_scope` before the `tf.compat.v1.get_variable()` call. NTcCsdS)aCreates a new variable with value `initial_value`. The new variable is added to the graph collections listed in `collections`, which defaults to `[GraphKeys.GLOBAL_VARIABLES]`. If `trainable` is `True` the variable is also added to the graph collection `GraphKeys.TRAINABLE_VARIABLES`. This constructor creates both a `variable` Op and an `assign` Op to set the variable to its initial value. Args: initial_value: A `Tensor`, or Python object convertible to a `Tensor`, which is the initial value for the Variable. The initial value must have a shape specified unless `validate_shape` is set to False. Can also be a callable with no argument that returns the initial value when called. In that case, `dtype` must be specified. (Note that initializer functions from init_ops.py must first be bound to a shape before being used here.) trainable: If `True`, also adds the variable to the graph collection `GraphKeys.TRAINABLE_VARIABLES`. This collection is used as the default list of variables to use by the `Optimizer` classes. Defaults to `True`, unless `synchronization` is set to `ON_READ`, in which case it defaults to `False`. collections: List of graph collections keys. The new variable is added to these collections. Defaults to `[GraphKeys.GLOBAL_VARIABLES]`. validate_shape: If `False`, allows the variable to be initialized with a value of unknown shape. If `True`, the default, the shape of `initial_value` must be known. caching_device: Optional device string describing where the Variable should be cached for reading. Defaults to the Variable's device. If not `None`, caches on another device. Typical use is to cache on the device where the Ops using the Variable reside, to deduplicate copying through `Switch` and other conditional statements. name: Optional name for the variable. Defaults to `'Variable'` and gets uniquified automatically. variable_def: `VariableDef` protocol buffer. If not `None`, recreates the Variable object with its contents, referencing the variable's nodes in the graph, which must already exist. The graph is not changed. `variable_def` and the other arguments are mutually exclusive. dtype: If set, initial_value will be converted to the given type. If `None`, either the datatype will be kept (if `initial_value` is a Tensor), or `convert_to_tensor` will decide. expected_shape: A TensorShape. If set, initial_value is expected to have this shape. import_scope: Optional `string`. Name scope to add to the `Variable.` Only used when initializing from protocol buffer. constraint: An optional projection function to be applied to the variable after being updated by an `Optimizer` (e.g. used to implement norm constraints or value constraints for layer weights). The function must take as input the unprojected Tensor representing the value of the variable and return the Tensor for the projected value (which must have the same shape). Constraints are not safe to use when doing asynchronous distributed training. use_resource: whether to use resource variables. synchronization: Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class `tf.VariableSynchronization`. By default the synchronization is set to `AUTO` and the current `DistributionStrategy` chooses when to synchronize. aggregation: Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class `tf.VariableAggregation`. shape: (optional) The shape of this variable. If None, the shape of `initial_value` will be used. When setting this argument to `tf.TensorShape(None)` (representing an unspecified shape), the variable can be assigned with values of different shapes. Raises: ValueError: If both `variable_def` and initial_value are specified. ValueError: If the initial value is not specified, or does not have a shape and `validate_shape` is `True`. RuntimeError: If eager execution is enabled. Nr")r@rVrNrWrXrYrOrZr[r\r]r^r_rLrMr`r"r"r#rrszVariableV1.__init__) r2r3r4r5r.r6r9r7rrrjrr"r"r"r#ri8s$t [ric@seZdZdZdkddZddZdldd Zdmd d Zd d ZddZ ddZ ddZ ddZ e ddZe ddZe ddZdnddZe ddZe d d!Zdod#d$Zdpd%d&Zdqd'd(Zdrd)d*Zdsd+d,Zdtd-d.Zdud/d0Zdvd1d2Zdwd3d4Zdxd5d6Zdyd7d8Zdzd9d:Zd{d;d<Z d|d=d>Z!d}d?d@Z"d~dAdBZ#dCdDZ$e%ddEdFdGZ&e'ddHdIZ(dJZ)e dKdLZ*e dMdNZ+e dOdPZ,e dQdRZ-e dSdTZ.e dUdVZ/e dWdXZ0e dYdZZ1dd[d\Z2d]d^Z3d_d`Z4dadbZ5dcddZ6dedfZ7dgdhZ8didjZ9dS)rz&Ref-based implementation of variables.NTcCsLd|_|r&|rtd|j|| dn"|j|||||||| | | | |d dS)aCreates a new variable with value `initial_value`. The new variable is added to the graph collections listed in `collections`, which defaults to `[GraphKeys.GLOBAL_VARIABLES]`. If `trainable` is `True` the variable is also added to the graph collection `GraphKeys.TRAINABLE_VARIABLES`. This constructor creates both a `variable` Op and an `assign` Op to set the variable to its initial value. Args: initial_value: A `Tensor`, or Python object convertible to a `Tensor`, which is the initial value for the Variable. The initial value must have a shape specified unless `validate_shape` is set to False. Can also be a callable with no argument that returns the initial value when called. In that case, `dtype` must be specified. (Note that initializer functions from init_ops.py must first be bound to a shape before being used here.) trainable: If `True`, also adds the variable to the graph collection `GraphKeys.TRAINABLE_VARIABLES`. This collection is used as the default list of variables to use by the `Optimizer` classes. Defaults to `True`, unless `synchronization` is set to `ON_READ`, in which case it defaults to `False`. collections: List of graph collections keys. The new variable is added to these collections. Defaults to `[GraphKeys.GLOBAL_VARIABLES]`. validate_shape: If `False`, allows the variable to be initialized with a value of unknown shape. If `True`, the default, the shape of `initial_value` must be known. caching_device: Optional device string describing where the Variable should be cached for reading. Defaults to the Variable's device. If not `None`, caches on another device. Typical use is to cache on the device where the Ops using the Variable reside, to deduplicate copying through `Switch` and other conditional statements. name: Optional name for the variable. Defaults to `'Variable'` and gets uniquified automatically. variable_def: `VariableDef` protocol buffer. If not `None`, recreates the Variable object with its contents, referencing the variable's nodes in the graph, which must already exist. The graph is not changed. `variable_def` and the other arguments are mutually exclusive. dtype: If set, initial_value will be converted to the given type. If `None`, either the datatype will be kept (if `initial_value` is a Tensor), or `convert_to_tensor` will decide. expected_shape: A TensorShape. If set, initial_value is expected to have this shape. import_scope: Optional `string`. Name scope to add to the `Variable.` Only used when initializing from protocol buffer. constraint: An optional projection function to be applied to the variable after being updated by an `Optimizer` (e.g. used to implement norm constraints or value constraints for layer weights). The function must take as input the unprojected Tensor representing the value of the variable and return the Tensor for the projected value (which must have the same shape). Constraints are not safe to use when doing asynchronous distributed training. synchronization: Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class `tf.VariableSynchronization`. By default the synchronization is set to `AUTO` and the current `DistributionStrategy` chooses when to synchronize. aggregation: Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class `tf.VariableAggregation`. shape: (optional) The shape of this variable. If None, the shape of `initial_value` will be used. When setting this argument to `tf.TensorShape(None)` (representing an unspecified shape), the variable can be assigned with values of different shapes. Raises: ValueError: If both `variable_def` and initial_value are specified. ValueError: If the initial value is not specified, or does not have a shape and `validate_shape` is `True`. RuntimeError: If eager execution is enabled. Tz6variable_def and initial_value are mutually exclusive.r) rVrNrWrXrYrOr[r\r^rLrMr`N)_in_graph_moderJ_init_from_proto_init_from_args)r@rVrNrWrXrYrOrZr[r\r]r^rLrMr`r"r"r#rrs&XzRefVariable.__init__cCsRtr6|js6d|j||jjtj|ddfSd|j||jjfSdS)Nz.T)is_reprz$) rrr rOrr[r numpy_textrvr?r"r"r#rs}szRefVariable.__repr__c  Cs|} |durtdt|}|dur.tjjg}t|tttfsRtd|t |f| durjt| sjtdt j |_ t|t j r||jj|_|j}t| | ||\} } }| |_| |_||_|rtjj|vrt|tjjg}ttrtdt|d|rgn|g}|rTt|}tjtjj t!"d|gdd }t #d |itd t$dp|}t|t j r||jj|_|j}tj%|d |d |_&| dur|r|j&'nt()} Wdn1s0YWdn1s0Yt*j+| |j&j,j-|d|_.Wdn1sH0Ynjtj%|d |d |_&|j&j/0durtd|| dur|r|j&'nt()} t*j+| |j&j,j-|d|_.|j.j1|_2|r|j&'}|3std|j&t*j4|j.t5||j&|dj/|_6|durXt$|"t7j8|j.dd|_9Wdn1sL0YnBt:|j.j/"t7j8|j.dd|_9Wdn1s0YWdn1s0Yt;||Wdn1s0Y||_dS)a> Creates a new variable from arguments. Args: initial_value: A `Tensor`, or Python object convertible to a `Tensor`, which is the initial value for the Variable. The initial value must have a shape specified unless `validate_shape` is set to False. Can also be a callable with no argument that returns the initial value when called. (Note that initializer functions from init_ops.py must first be bound to a shape before being used here.) trainable: If `True`, also adds the variable to the graph collection `GraphKeys.TRAINABLE_VARIABLES`. This collection is used as the default list of variables to use by the `Optimizer` classes. Defaults to `True`, unless `synchronization` is set to `ON_READ`, in which case it defaults to `False`. collections: List of graph collections keys. The new variable is added to these collections. Defaults to `[GraphKeys.GLOBAL_VARIABLES]`. validate_shape: If `False`, allows the variable to be initialized with a value of unknown shape. If `True`, the default, the shape of `initial_value` must be known. caching_device: Optional device string or function describing where the Variable should be cached for reading. Defaults to the Variable's device. If not `None`, caches on another device. Typical use is to cache on the device where the Ops using the Variable reside, to deduplicate copying through `Switch` and other conditional statements. name: Optional name for the variable. Defaults to `'Variable'` and gets uniquified automatically. dtype: If set, initial_value will be converted to the given type. If None, either the datatype will be kept (if initial_value is a Tensor) or float32 will be used (if it is a Python object convertible to a Tensor). expected_shape: Deprecated. Ignored. constraint: An optional projection function to be applied to the variable after being updated by an `Optimizer` (e.g. used to implement norm constraints or value constraints for layer weights). The function must take as input the unprojected Tensor representing the value of the variable and return the Tensor for the projected value (which must have the same shape). Constraints are not safe to use when doing asynchronous distributed training. synchronization: Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class `tf.VariableSynchronization`. By default the synchronization is set to `AUTO` and the current `DistributionStrategy` chooses when to synchronize. aggregation: Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class `tf.VariableAggregation`. shape: (optional) The shape of this variable. If None, the shape of `initial_value` will be used. When setting this argument to `tf.TensorShape(None)` (representing an unspecified shape), the variable can be assigned with values of different shapes. Raises: ValueError: If the initial value is not specified, or does not have a shape and `validate_shape` is `True`. RuntimeError: If lifted into the eager context. Nz initial_value must be specified.z]collections argument to Variable constructor must be a list, tuple, or set. Got %s of type %sz-The `constraint` argument must be a callable.zReference variables are not supported when eager execution is enabled. Please run `tf.compat.v1.enable_resource_variables()` to switch to resource variables.rjzloc:@%s)r)list_class InitializerrV)rOr[rOzInitializer for variable %s is from inside a control-flow construct, such as a loop or conditional. When creating a variable inside a loop or conditional, use a lambda as the initializer.z-initial_value must have a shape specified: %s)rXread)?rJcallabler GraphKeysGLOBAL_VARIABLESrBrtuplesettyperb _graph_keyrCheckpointInitialValue_maybe_initialize_trackablecheckpoint_position restore_uid _update_uid wrapped_valuerP_synchronization _aggregation _trainableTRAINABLE_VARIABLESr~rr RuntimeError name_scopename_from_scope_namer AttrValue ListValueras_bytes _attr_scoperconvert_to_tensor_initial_valuerr unknown_shapervariable_op_v2r[ base_dtype _variabler_get_control_flow_contextrO_nameis_fully_definedr-_try_guard_against_uninitialized_dependencies_initializer_opr identity _snapshot colocate_withadd_to_collections_caching_devicer _constraint)r@rVrNrWrXrYrOr[r\r^rLrMr`r init_from_fnZ true_nameattrZinitial_value_shaper"r"r#r sD         B *       4R,zRefVariable._init_from_argscCst|tjsJt}|tj|j|d|_|jj |_ |tj|j |d|_ t |dr||jr||tj|j|d|_nd|_t|j|j|j|j\}}}||_||_||_|tj|j|d|_|drtj|j|d|_nd|_d|_d|_dS)aRecreates the Variable object from a `VariableDef` protocol buffer. Args: variable_def: `VariableDef` protocol buffer, describing a variable whose nodes already exists in the graph. import_scope: Optional `string`. Name scope to add. rinitial_value_nameNr)rr]) rBr VariableDefrrbas_graph_elementr variable_namer1rOr3initializer_namer6hasattrr?r-rPrLrMrNr!r"r# snapshot_namer8HasFieldrjrrrr;r<)r@rZr]grLrMrNr"r"r#r EsV    zRefVariable._init_from_protocCs|jS)z1Conversion function for Graph.as_graph_element().r1r?r"r"r#_as_graph_elementrszRefVariable._as_graph_elementcCs|jS)rt)r8r?r"r"r#r>vszRefVariable.valuecCstj|jddS)rurr)r r7r1r?r"r"r#rvs zRefVariable.read_valuecCs|jS)aReturns a reference to this variable. You usually do not need to call this method as all ops that need a reference to the variable call it automatically. Returns is a `Tensor` which holds a reference to the variable. You can assign a new value to the variable by passing the tensor to an assign op. See `tf.Variable.value` if you want to get the value of the variable. Returns: A `Tensor` that is a reference to the variable. rHr?r"r"r#rszRefVariable._refcCs ||||dSrw)rryr>rxr"r"r#ryszRefVariable.set_shapecCs|jSr&)r#r?r"r"r#rNszRefVariable.trainablecCs|jSr&)r!r?r"r"r#rLszRefVariable.synchronizationcCs|jSr&)r"r?r"r"r#rMszRefVariable.aggregationcCs|jj|dS)rz)r|)r1r}r{r"r"r#r}szRefVariable.evalcCs|jS)r)r-r?r"r"r#rVs zRefVariable.initial_valuecCs|jS)r)r<r?r"r"r#r^szRefVariable.constraintFcCs"tj|j|||d}|r|S|jS)a(Assigns a new value to the variable. This is essentially a shortcut for `assign(self, value)`. Args: value: A `Tensor`. The new value for this variable. use_locking: If `True`, use locking during the assignment. name: The name of the operation to be created read_value: if True, will return something which evaluates to the new value of the variable; if False will return the assign op. Returns: A `Tensor` that will hold the new value of this variable after the assignment has completed. rrO)rrr1r)r@r>rrOrvrr"r"r#rs  zRefVariable.assigncCs"tj|j|||d}|r|S|jS)a&Adds a value to this variable. This is essentially a shortcut for `assign_add(self, delta)`. Args: delta: A `Tensor`. The value to add to this variable. use_locking: If `True`, use locking during the operation. name: The name of the operation to be created read_value: if True, will return something which evaluates to the new value of the variable; if False will return the assign op. Returns: A `Tensor` that will hold the new value of this variable after the addition has completed. rJ)rrr1rr@rrrOrvrr"r"r#rs  zRefVariable.assign_addcCs"tj|j|||d}|r|S|jS)a6Subtracts a value from this variable. This is essentially a shortcut for `assign_sub(self, delta)`. Args: delta: A `Tensor`. The value to subtract from this variable. use_locking: If `True`, use locking during the operation. name: The name of the operation to be created read_value: if True, will return something which evaluates to the new value of the variable; if False will return the assign op. Returns: A `Tensor` that will hold the new value of this variable after the subtraction has completed. rJ)rrr1rrKr"r"r#rs  zRefVariable.assign_subcCs2t|tjstd|tj|j|j|j||dS)aSubtracts `tf.IndexedSlices` from this variable. Args: sparse_delta: `tf.IndexedSlices` to be subtracted from this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered subtraction has completed. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. %sparse_delta is not IndexedSlices: %srJ) rBr IndexedSlicesrrrr1rvaluesrr"r"r#r3s  zRefVariable.scatter_subcCs2t|tjstd|tj|j|j|j||dS)aAdds `tf.IndexedSlices` to this variable. Args: sparse_delta: `tf.IndexedSlices` to be added to this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered addition has completed. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. rLrJ) rBrrMrrrr1rrNrr"r"r#rKs  zRefVariable.scatter_addcCs2t|tjstd|tj|j|j|j||dS)aUpdates this variable with the max of `tf.IndexedSlices` and itself. Args: sparse_delta: `tf.IndexedSlices` to use as an argument of max with this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered maximization has completed. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. rLrJ) rBrrMrrrr1rrNrr"r"r#rcs  zRefVariable.scatter_maxcCs2t|tjstd|tj|j|j|j||dS)aUpdates this variable with the min of `tf.IndexedSlices` and itself. Args: sparse_delta: `tf.IndexedSlices` to use as an argument of min with this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered minimization has completed. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. rLrJ) rBrrMrrrr1rrNrr"r"r#r|s  zRefVariable.scatter_mincCs2t|tjstd|tj|j|j|j||dS)aMultiply this variable by `tf.IndexedSlices`. Args: sparse_delta: `tf.IndexedSlices` to multiply this variable by. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered multiplication has completed. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. rLrJ) rBrrMrrrr1rrNrr"r"r#rs  zRefVariable.scatter_mulcCs2t|tjstd|tj|j|j|j||dS)aDivide this variable by `tf.IndexedSlices`. Args: sparse_delta: `tf.IndexedSlices` to divide this variable by. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered division has completed. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. rLrJ) rBrrMrrrr1rrNrr"r"r#rs  zRefVariable.scatter_divcCs2t|tjstd|tj|j|j|j||dS)aAssigns `tf.IndexedSlices` to this variable. Args: sparse_delta: `tf.IndexedSlices` to be assigned to this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered assignment has completed. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. rLrJ) rBrrMrrrr1rrNrr"r"r#rs  zRefVariable.scatter_updatecCstj||j|j||dS)atAssigns `tf.IndexedSlices` to this variable batch-wise. Analogous to `batch_gather`. This assumes that this variable and the sparse_delta IndexedSlices have a series of leading dimensions that are the same for all of them, and the updates are performed on the last dimension of indices. In other words, the dimensions should be the following: `num_prefix_dims = sparse_delta.indices.ndims - 1` `batch_dim = num_prefix_dims + 1` `sparse_delta.updates.shape = sparse_delta.indices.shape + var.shape[ batch_dim:]` where `sparse_delta.updates.shape[:num_prefix_dims]` `== sparse_delta.indices.shape[:num_prefix_dims]` `== var.shape[:num_prefix_dims]` And the operation performed can be expressed as: `var[i_1, ..., i_n, sparse_delta.indices[i_1, ..., i_n, j]] = sparse_delta.updates[ i_1, ..., i_n, j]` When sparse_delta.indices is a 1D tensor, this operation is equivalent to `scatter_update`. To avoid this operation one can looping over the first `ndims` of the variable and using `scatter_update` on the subtensors that result of slicing the first dimension. This is a valid option for `ndims = 1`, but less efficient than this implementation. Args: sparse_delta: `tf.IndexedSlices` to be assigned to this variable. use_locking: If `True`, use locking during the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered assignment has completed. Raises: TypeError: if `sparse_delta` is not an `IndexedSlices`. rJ)rrrrNrr"r"r#rs-z RefVariable.batch_scatter_updatecCstj|j||d|dS)aApplies sparse subtraction to individual values or slices in a Variable. `ref` is a `Tensor` with rank `P` and `indices` is a `Tensor` of rank `Q`. `indices` must be integer tensor, containing indices into `ref`. It must be shape `[d_0, ..., d_{Q-2}, K]` where `0 < K <= P`. The innermost dimension of `indices` (with length `K`) corresponds to indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th dimension of `ref`. `updates` is `Tensor` of rank `Q-1+P-K` with shape: ``` [d_0, ..., d_{Q-2}, ref.shape[K], ..., ref.shape[P-1]]. ``` For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this: ```python ref = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8]) indices = tf.constant([[4], [3], [1] ,[7]]) updates = tf.constant([9, 10, 11, 12]) op = ref.scatter_nd_sub(indices, updates) with tf.compat.v1.Session() as sess: print sess.run(op) ``` The resulting update to ref would look like this: [1, -9, 3, -6, -6, 6, 7, -4] See `tf.scatter_nd` for more details about how to make updates to slices. Args: indices: The indices to be used in the operation. updates: The values to be used in the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered subtraction has completed. TrJ)rrr1rr"r"r#r s. zRefVariable.scatter_nd_subcCstj|j||d|dS)aApplies sparse addition to individual values or slices in a Variable. `ref` is a `Tensor` with rank `P` and `indices` is a `Tensor` of rank `Q`. `indices` must be integer tensor, containing indices into `ref`. It must be shape `[d_0, ..., d_{Q-2}, K]` where `0 < K <= P`. The innermost dimension of `indices` (with length `K`) corresponds to indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th dimension of `ref`. `updates` is `Tensor` of rank `Q-1+P-K` with shape: ``` [d_0, ..., d_{Q-2}, ref.shape[K], ..., ref.shape[P-1]]. ``` For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this: ```python ref = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8]) indices = tf.constant([[4], [3], [1] ,[7]]) updates = tf.constant([9, 10, 11, 12]) add = ref.scatter_nd_add(indices, updates) with tf.compat.v1.Session() as sess: print sess.run(add) ``` The resulting update to ref would look like this: [1, 13, 3, 14, 14, 6, 7, 20] See `tf.scatter_nd` for more details about how to make updates to slices. Args: indices: The indices to be used in the operation. updates: The values to be used in the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered addition has completed. TrJ)rrr1rr"r"r#rB s. zRefVariable.scatter_nd_addcCstj|j||d|dS)aApplies sparse assignment to individual values or slices in a Variable. `ref` is a `Tensor` with rank `P` and `indices` is a `Tensor` of rank `Q`. `indices` must be integer tensor, containing indices into `ref`. It must be shape `[d_0, ..., d_{Q-2}, K]` where `0 < K <= P`. The innermost dimension of `indices` (with length `K`) corresponds to indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th dimension of `ref`. `updates` is `Tensor` of rank `Q-1+P-K` with shape: ``` [d_0, ..., d_{Q-2}, ref.shape[K], ..., ref.shape[P-1]]. ``` For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this: ```python ref = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8]) indices = tf.constant([[4], [3], [1] ,[7]]) updates = tf.constant([9, 10, 11, 12]) op = ref.scatter_nd_update(indices, updates) with tf.compat.v1.Session() as sess: print sess.run(op) ``` The resulting update to ref would look like this: [1, 11, 3, 10, 9, 6, 7, 12] See `tf.scatter_nd` for more details about how to make updates to slices. Args: indices: The indices to be used in the operation. updates: The values to be used in the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered assignment has completed. TrJ)rrr1rr"r"r#rs s. zRefVariable.scatter_nd_updatecCstj|j||d|dS)aUpdates this variable with the max of `tf.IndexedSlices` and itself. `ref` is a `Tensor` with rank `P` and `indices` is a `Tensor` of rank `Q`. `indices` must be integer tensor, containing indices into `ref`. It must be shape `[d_0, ..., d_{Q-2}, K]` where `0 < K <= P`. The innermost dimension of `indices` (with length `K`) corresponds to indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th dimension of `ref`. `updates` is `Tensor` of rank `Q-1+P-K` with shape: ``` [d_0, ..., d_{Q-2}, ref.shape[K], ..., ref.shape[P-1]]. ``` See `tf.scatter_nd` for more details about how to make updates to slices. Args: indices: The indices to be used in the operation. updates: The values to be used in the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered addition has completed. TrJ)rscatter_nd_maxr1rr"r"r#rO s zRefVariable.scatter_nd_maxcCstj|j||d|dS)aUpdates this variable with the min of `tf.IndexedSlices` and itself. `ref` is a `Tensor` with rank `P` and `indices` is a `Tensor` of rank `Q`. `indices` must be integer tensor, containing indices into `ref`. It must be shape `[d_0, ..., d_{Q-2}, K]` where `0 < K <= P`. The innermost dimension of `indices` (with length `K`) corresponds to indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th dimension of `ref`. `updates` is `Tensor` of rank `Q-1+P-K` with shape: ``` [d_0, ..., d_{Q-2}, ref.shape[K], ..., ref.shape[P-1]]. ``` See `tf.scatter_nd` for more details about how to make updates to slices. Args: indices: The indices to be used in the operation. updates: The values to be used in the operation. name: the name of the operation. Returns: A `Tensor` that will hold the new value of this variable after the scattered addition has completed. TrJ)rscatter_nd_minr1rr"r"r#rP s zRefVariable.scatter_nd_minc Cs$tj|||||||||| | d S)N) rbeginendstridesr>rO begin_maskend_mask ellipsis_mask new_axis_maskshrink_axis_mask)r strided_slice_assignr) r@rQrRrSr>rOrTrUrVrWrXr"r"r#_strided_slice_assign sz!RefVariable._strided_slice_assignrcCstj|j|dS)r)r)rrr1rr"r"r#r szRefVariable.count_up_tocCsB|}|r*||js*td|j|jjf|r6|S|SdS)rMIncompatible type conversion requested to type '%s' for variable of type '%s'Nrrr"r"r#r s z%RefVariable._TensorConversionFunctionrcCs|jS)rr3r?r"r"r#rO$ szRefVariable.namecCs|jS)r)r6r?r"r"r#r) szRefVariable.initializercCs|jjS)r)r1rr?r"r"r#r. szRefVariable.devicecCs|jjS)r)r1r[r?r"r"r#r[3 szRefVariable.dtypecCs|jjS)r)r1rr?r"r"r#r8 szRefVariable.opcCs|jjS)r)r1rr?r"r"r#r= szRefVariable.graphcCsdSzBThe `tf.distribute.Strategy` that this variable was created under.Nr"r?r"r"r#_distribute_strategyB sz RefVariable._distribute_strategycCs |jS)r)r1rr?r"r"r#r`G szRefVariable.shapecCs|dus|jj|rt}t|jj||_|jdurLt|jj||_ |j |_ |j j |_ |j j |_ t|jj||_t|jj||_|jr|j|jj|d|SdSdS)rN)r)r1rOrrr@rrrBr-r?rNrLr>rMrrCr8rErr MergeFromr)r@rvar_defr"r"r#rP s0        zRefVariable.to_protocCsttjdd||S)NzVariable += will be deprecated. Use variable.assign_add if you want assignment to the variable value or 'x = x + y' if you want a new python Tensor object.r/logging log_first_nWARNrDr"r"r#__iadd__p s zRefVariable.__iadd__cCsttjdd||S)NzVariable -= will be deprecated. Use variable.assign_sub if you want assignment to the variable value or 'x = x - y' if you want a new python Tensor object.r/rarDr"r"r#__isub__w s zRefVariable.__isub__cCsttjdd||S)NzVariable *= will be deprecated. Use `var.assign(var * other)` if you want assignment to the variable value or `x = x * y` if you want a new python Tensor object.r/rarDr"r"r#__imul__~ s zRefVariable.__imul__cCsttjdd||SNzVariable /= will be deprecated. Use `var.assign(var / other)` if you want assignment to the variable value or `x = x / y` if you want a new python Tensor object.r/rarDr"r"r#__idiv__ s zRefVariable.__idiv__cCsttjdd||SrhrarDr"r"r# __itruediv__ s zRefVariable.__itruediv__cCsttjdd||SrhrarDr"r"r# __irealdiv__ s zRefVariable.__irealdiv__cCsttjdd||S)NzVariable **= will be deprecated. Use `var.assign(var ** other)` if you want assignment to the variable value or `x = x ** y` if you want a new python Tensor object.r/rarDr"r"r#__ipow__ s zRefVariable.__ipow__)NNNTNNNNNNNNNN) NNNTNNNNNNNN)N)N)FNT)FNT)FNT)FN)FN)FN)FN)FN)FN)FN)FN)N)N)N)N)N)NNF)N):r2r3r4r5rrrsr r rIr>rvrryrrNrLrMr}rVr^rrrrrrrrrrrrrrrOrPrZrrrrrrOrrr[rrr^r`rrerfrgrirjrkrlr"r"r"r#r s o  @ -                  4 1 1 1 ! !           rcCs8t|tjstd|t|jidr*|St||idS)aAttempt to guard against dependencies on uninitialized variables. Replace references to variables in `initial_value` with references to the variable's initialized values. The initialized values are essentially conditional TensorFlow graphs that return a variable's value if it is initialized or its `initial_value` if it hasn't been initialized. This replacement is done on a best effort basis: - If the `initial_value` graph contains cycles, we don't do any replacements for that graph. - If the variables that `initial_value` depends on are not present in the `GLOBAL_VARIABLES` or `LOCAL_VARIABLES` we don't replace them. In these cases, it is up to the caller to ensure that the `initial_value` graph uses initialized variables or that they guard access to variables using their `initialized_value` method. Args: name: Variable name. initial_value: `Tensor`. The initial value. Returns: A `Tensor` suitable to initialize a variable. Raises: TypeError: If `initial_value` is not a `Tensor`. z&initial_value needs to be a Tensor: %s)state)op_cache)rBrrr _has_cycler_safe_initial_value_from_tensor)rOrVr"r"r#r5 s   r5r1cCsn||jt}|tkrdS|tkr&dSt||j<tdd|jD|jD]}t ||rJdSqJt||j<dS)z5Detect cycles in the dependencies of `initial_value`.TFcss|] }|jVqdSr&rrr"r"r#r rUz_has_cycle..) getrO_UNKNOWN_STARTED _FINISHED itertoolschainrcontrol_inputsro)rrmZop_staterr"r"r#ro s   rocCs<|j}||j}|dur0t|||}|||j<|j|jS)aWReplace dependencies on variables with their initialized values. Args: name: Variable name. tensor: A `Tensor`. The tensor to replace. op_cache: A dict mapping operation names to `Operation`s. Used to memoize the results so as to avoid creating redundant operations. Returns: A `Tensor` compatible with `tensor`. Any inputs that lead to variable values will be replaced with a corresponding graph that uses the variable's initialized values. This is done on a best-effort basis. If no modifications need to be made then `tensor` will be returned unchanged. N)rrrrO_safe_initial_value_from_opoutputs value_index)rOtensorrnrnew_opr"r"r#rp s    rpc Cs|jj}|dvr|S|dvr6t|}|dur0|S|jSd}g}|jD]&}t|||}|||ph||k}qD|r|} | dkrd} |jjd|} | dd} |jj | ||j | |jj d S|S) aVReplace dependencies on variables with their initialized values. Args: name: Variable name. op: An `Operation`. The operation to replace. op_cache: A dict mapping operation names to `Operation`s. Used to memoize the results so as to avoid creating redundant operations. Returns: An `Operation` compatible with `op`. Any inputs that lead to variable values will be replaced with a corresponding graph that uses the variable's initialized values. This is done on a best-effort basis. If no modifications need to be made then `op` will be returned unchanged. )IsVariableInitializedVarIsInitializedOpReadVariableOpIf)rj VariableV2 VarHandleOpNF RefSwitchSwitchr r)rOattrs) node_defr$_find_initialized_value_for_variablerrprrOreplacer create_op _output_typesr>) rOrrnop_typermodifiedZ new_op_inputsop_inputZ new_op_inputZ new_op_typeZ new_op_namer"r"r#ry s4    rycCsvz\|jj|jjdg}tjjtjjfD]2}|j|D] }|j|vr6|WSq6q&Wnt ypYdS0dS)a-Find the initialized value for a variable op. To do so, lookup the variable op in the variables collection. Args: variable_op: A variable `Operation`. Returns: A `Tensor` representing the initialized value for the variable or `None` if the initialized value could not be found. z:0N) rrOrrrLOCAL_VARIABLESrget_collectionrr) variable_op var_namescollection_namevarr"r"r#r( s   rc@seZdZdZddZddZddZdd Zd d Zd d Z e d)ddZ e ddZ e ddZe ddZe ddZddZddZddZd d!Zd*d#d$Zd+d%d&Zd,d'd(ZdS)-PartitionedVariableaA container for partitioned `Variable` objects. @compatibility(eager) `tf.PartitionedVariable` is not compatible with eager execution. Use `tf.Variable` instead which is compatible with both eager execution and graph construction. See [the TensorFlow Eager Execution guide](https://www.tensorflow.org/guide/eager#variables_and_optimizers) for details on how variables work in eager execution. @end_compatibility cCst|ttfstd|t|ttfs4td|tdd|DsRtd||s^td|D]v}tdd|Dstdd d |Dt|t|krtd ||f|j|krbtd |t |jgfqbt |d dd|_ ||_ ||_ ||_||_d|_dS)aSCreates a new partitioned variable wrapper. Variables passed via the variable_list must contain a save_slice_info field. Concatenation and iteration is in lexicographic order according to the var_offset property of the save_slice_info. Args: name: String. Overall name of the variables. shape: List of integers. Overall shape of the variables. dtype: Type of the variables. variable_list: List of `Variable` that comprise this partitioned variable. partitions: List of integers. Number of partitions for each dimension. Raises: TypeError: If `variable_list` is not a list of `Variable` objects, or `partitions` is not a list. ValueError: If `variable_list` is empty, or the `Variable` shape information does not match `shape`, or `partitions` has invalid values. z(variable_list is not a list or tuple: %sz%partitions is not a list or tuple: %scss|]}|dkVqdSr/Nr"rpr"r"r#rg rUz/PartitionedVariable.__init__..z%partition values must be positive: %szvariable_list may not be emptycss|]}|duVqdSr&)rrrr"r"r#rn rUz7All variables must have a save_slice_info available: %scSsg|] }|jqSr"rrr"r"r#rq rUz0PartitionedVariable.__init__..z(len(shape) != len(partitions): %s vs. %szIAll variables' full shapes must match shape: %s; but full shapes were: %scSs |jSr&)rr)rr"r"r#rTz rUz.PartitionedVariable.__init__..)keyN)rBrrrallrJlenrrstrsorted_variable_listr3_shape_dtype _partitions _as_tensor)r@rOr`r[ variable_list partitionsrr"r"r#rrN sB   zPartitionedVariable.__init__cCs t|jS)zDReturn an iterable for accessing the underlying partition Variables.)rrr?r"r"r#r szPartitionedVariable.__iter__cCs*t|}|dkr td|t|jS)Nr/z-Cannot get a length for %d > 1 partition axes)r_partition_axesrJr)r@Znum_partition_axesr"r"r#__len__ s  zPartitionedVariable.__len__cCs2tdd|jDrdgSddt|jDSdS)Ncss|]}|dkVqdSrr"rr"r"r#r rUz6PartitionedVariable._partition_axes..rcSsg|]\}}|dkr|qS)r/r")rrrr"r"r#r rUz7PartitionedVariable._partition_axes..)rr enumerater?r"r"r#r sz#PartitionedVariable._partition_axescCst|jdkrNtd&tj|jd|jdWdS1sD0Y|}t|dkrrtdt ||d}t|jdt |j|}Wdn1s0Ytd tj||jdWdS1s0YdS)a]Returns the overall concatenated value as a `Tensor`. This is different from using the partitioned variable directly as a tensor (through tensor conversion and `as_tensor`) in that it creates a new set of operations that keeps the control dependencies from its scope. Returns: `Tensor` containing the concatenated value. r/NrrzcCannot concatenate along more than one dimension: %s. Multi-axis partition concat is not supportedz/ConcatPartitions/) rrrr&r r7r3rrrconcat)r@partition_axes partition_ix concatenatedr"r"r#_concat s  4 , zPartitionedVariable._concatcCs6td|WdS1s(0YdS)a"Returns the overall concatenated value as a `Tensor`. The returned tensor will not inherit the control dependencies from the scope where the value is used, which is similar to getting the value of `Variable`. Returns: `Tensor` containing the concatenated value. N)rcontrol_dependenciesrr?r"r"r# as_tensor s zPartitionedVariable.as_tensorNFcCsH|}|dur.||js.td|j|jjf|rPartitionedVariable doesn't support being used as a reference.)rr[rJrOrrrr"r"r#r s z-PartitionedVariable._TensorConversionFunctioncCs|jSr&r\r?r"r"r#rO szPartitionedVariable.namecCs|jSr&)rr?r"r"r#r[ szPartitionedVariable.dtypecCs|Sr&)rr?r"r"r#r` szPartitionedVariable.shapecCsdSr]r"r?r"r"r#r^ sz(PartitionedVariable._distribute_strategycCs|jSr&)rr?r"r"r#r szPartitionedVariable.get_shapecCs|jSr&)rr?r"r"r#_get_variable_list sz&PartitionedVariable._get_variable_listcCs|jSr&)rr?r"r"r#_get_partitions sz#PartitionedVariable._get_partitionscs|}t|dkr$tdt|t|trJt|t|jksDJ|nFt|trddd|Dn,|dfdd|jD}tj ||dfddt |jD}|S) Nr/zpCannot do assign action along more than one dimension: %s. Multi-axis partition assign action is not supported cSsg|]}|qSr"r")rZvar_partr"r"r#r rUz8PartitionedVariable._apply_assign_fn..rcsg|]}t|jqSr")r dimension_valuer`)rr)rr"r#r s)axiscsg|]\}}||qSr"r")ridxr) assign_fn value_listr"r#r s) rrrrrBrrrr splitr)r@rr>rZsize_splits_listop_listr")rrrr#_apply_assign_fn s*     z$PartitionedVariable._apply_assign_fnTcs2fdd}|||}r$|Sdd|DS)Ncs|j|dSN)rrOrv)rrZr_valuerOrvrr"r#rT sz,PartitionedVariable.assign..cSsg|] }|jqSr"rqrrr"r"r#r rUz.PartitionedVariable.assign..rr@r>rrOrvrZ assign_listr"rr#r s  zPartitionedVariable.assigncs2fdd}|||}r$|Sdd|DS)Ncs|j|dSr)rrrr"r#rT sz0PartitionedVariable.assign_add..cSsg|] }|jqSr"rqrr"r"r#r rUz2PartitionedVariable.assign_add..rrr"rr#r s  zPartitionedVariable.assign_addcs2fdd}|||}r$|Sdd|DS)Ncs|j|dSr)rrrr"r#rT sz0PartitionedVariable.assign_sub..cSsg|] }|jqSr"rqrr"r"r#r rUz2PartitionedVariable.assign_sub..rrr"rr#r s  zPartitionedVariable.assign_sub)NNF)FNT)FNT)FNT)r2r3r4r5rrrrrrrrrrrOr[r`r^rrrrrrrr"r"r"r#rB s0 5        rglobal_variablescCsttjj|S)aReturns global variables. Global variables are variables that are shared across machines in a distributed environment. The `Variable()` constructor or `get_variable()` automatically adds new variables to the graph collection `GraphKeys.GLOBAL_VARIABLES`. This convenience function returns the contents of that collection. An alternative to global variables are local variables. See `tf.compat.v1.local_variables` @compatibility(TF2) Not compatible with eager execution and `tf.function`. In particular, Graph collections are deprecated in TF2. Instead please create a [tf.Module](https://www.tensorflow.org/guide/intro_to_modules) container for all your model state, including variables. You can then list all the variables in your `tf.Module` through the `variables` attribute. @end_compatibility Args: scope: (Optional.) A string. If supplied, the resulting list is filtered to include only items whose `name` attribute matches `scope` using `re.match`. Items without a `name` attribute are never returned if a scope is supplied. The choice of `re.match` means that a `scope` without special tokens filters by prefix. Returns: A list of `Variable` objects. )rrrrscoper"r"r#r s all_variablesz 2017-03-02z'Please use tf.global_variables instead.cCstS)z,Use `tf.compat.v1.global_variables` instead.)rr"r"r"r#rB scCs ttjj|ttjj|S)aReturns all variables and `SaveableObject`s that must be checkpointed. Args: scope: (Optional.) A string. If supplied, the resulting list is filtered to include only items whose `name` attribute matches `scope` using `re.match`. Items without a `name` attribute are never returned if a scope is supplied. The choice of `re.match` means that a `scope` without special tokens filters by prefix. Returns: A list of `Variable` and `SaveableObject` to be checkpointed )rrrrSAVEABLE_OBJECTSrr"r"r#_all_saveable_objectsI srlocal_variablescCsttjj|S)aReturns local variables. Local variables - per process variables, usually not saved/restored to checkpoint and used for temporary or intermediate values. For example, they can be used as counters for metrics computation or number of epochs this machine has read data. The `tf.contrib.framework.local_variable()` function automatically adds the new variable to `GraphKeys.LOCAL_VARIABLES`. This convenience function returns the contents of that collection. An alternative to local variables are global variables. See `tf.compat.v1.global_variables` Args: scope: (Optional.) A string. If supplied, the resulting list is filtered to include only items whose `name` attribute matches `scope` using `re.match`. Items without a `name` attribute are never returned if a scope is supplied. The choice of `re.match` means that a `scope` without special tokens filters by prefix. Returns: A list of local `Variable` objects. )rrrrrr"r"r#r[ smodel_variablescCsttjj|S)aReturns all variables in the MODEL_VARIABLES collection. Args: scope: (Optional.) A string. If supplied, the resulting list is filtered to include only items whose `name` attribute matches `scope` using `re.match`. Items without a `name` attribute are never returned if a scope is supplied. The choice of `re.match` means that a `scope` without special tokens filters by prefix. Returns: A list of local Variable objects. )rrrMODEL_VARIABLESrr"r"r#rw strainable_variablescCsttjj|S)aReturns all variables created with `trainable=True`. When passed `trainable=True`, the `Variable()` constructor automatically adds new variables to the graph collection `GraphKeys.TRAINABLE_VARIABLES`. This convenience function returns the contents of that collection. @compatibility(TF2) Not compatible with eager execution and `tf.function`. In particular, Graph collections are deprecated in TF2. Instead please create a `tf.Module` container for all your model state, including variables. You can then list all the trainable variables in your `tf.Module` through the `trainable_variables` attribute. @end_compatibility Args: scope: (Optional.) A string. If supplied, the resulting list is filtered to include only items whose `name` attribute matches `scope` using `re.match`. Items without a `name` attribute are never returned if a scope is supplied. The choice of `re.match` means that a `scope` without special tokens filters by prefix. Returns: A list of Variable objects. )rrrr$rr"r"r#r smoving_average_variablescCsttjj|S)aReturns all variables that maintain their moving averages. If an `ExponentialMovingAverage` object is created and the `apply()` method is called on a list of variables, these variables will be added to the `GraphKeys.MOVING_AVERAGE_VARIABLES` collection. This convenience function returns the contents of that collection. Args: scope: (Optional.) A string. If supplied, the resulting list is filtered to include only items whose `name` attribute matches `scope` using `re.match`. Items without a `name` attribute are never returned if a scope is supplied. The choice of `re.match` means that a `scope` without special tokens filters by prefix. Returns: A list of Variable objects. )rrrMOVING_AVERAGE_VARIABLESrr"r"r#r szinitializers.variablesvariables_initializerinitcCs2|r&ts&tjdd|Dd|iStj|dS)a]Returns an Op that initializes a list of variables. After you launch the graph in a session, you can run the returned Op to initialize all the variables in `var_list`. This Op runs all the initializers of the variables in `var_list` in parallel. Calling `initialize_variables()` is equivalent to passing the list of initializers to `Group()`. If `var_list` is empty, however, the function still returns an Op that can be run. That Op just has no effect. @compatibility(TF2) In TF2, variables are initialized immediately when they are created. There is no longer a need to run variable initializers before using them. @end_compatibility Args: var_list: List of `Variable` objects to initialize. name: Optional name for the returned operation. Returns: An Op that run the initializers of all the specified variables. cSsg|] }|jqSr")rrr"r"r#r rUz)variables_initializer..rOr)rrr groupno_opvar_listrOr"r"r#r s initialize_variablesz'Use `tf.variables_initializer` instead.cCs t||dS)z)See `tf.compat.v1.variables_initializer`.r)rrr"r"r#r szinitializers.global_variablesglobal_variables_initializercCstrtjddSttS)aReturns an Op that initializes global variables. This is just a shortcut for `variables_initializer(global_variables())` @compatibility(TF2) In TF2, variables are initialized immediately when they are created. There is no longer a need to run variable initializers before using them. @end_compatibility Returns: An Op that initializes global variables in the graph. rr)rrr rrrr"r"r"r#r s initialize_all_variablesz.Use `tf.global_variables_initializer` instead.cCstS)z0See `tf.compat.v1.global_variables_initializer`.)rr"r"r"r#r szinitializers.local_variableslocal_variables_initializercCstrtjddSttS)aReturns an Op that initializes all local variables. This is just a shortcut for `variables_initializer(local_variables())` @compatibility(TF2) In TF2, variables are initialized immediately when they are created. There is no longer a need to run variable initializers before using them. @end_compatibility Returns: An Op that initializes all local variables in the graph. rr)rrr rrrr"r"r"r#r s initialize_local_variablesz-Use `tf.local_variables_initializer` instead.cCstS)z/See `tf.compat.v1.local_variables_initializer`.)rr"r"r"r#r srcCs t|S)zTests if a variable has been initialized. Args: variable: A `Variable`. Returns: Returns a scalar boolean Tensor, `True` if the variable has been initialized, `False` otherwise. rr)variabler"r"r#r s assert_variables_initializedc Cs|durtt}|sHg}tD]}|jdvr(||jdq(|sPdSg}|D]D}t|j $|t j |ddWdqX1s0YqXt |dkr|dSt |SdS)aReturns an Op to check if variables are initialized. NOTE: This function is obsolete and will be removed in 6 months. Please change your implementation to use `report_uninitialized_variables()`. When run, the returned Op will raise the exception `FailedPreconditionError` if any of the variables has not yet been initialized. Note: This function is implemented by trying to fetch the values of the variables. If one of the variables is not initialized a message may be logged by the C++ runtime. This is expected. Args: var_list: List of `Variable` objects to check. Defaults to the value of `global_variables().` Returns: An Op, or None if there are no variables. NrjrZAutoReloadVariablerF)optimizer/)rrrrbget_operationsrrrzr9rr rank_internalrstack)rrranksrr"r"r#r( s   4 report_uninitialized_variablesc Cs@|durHtt}|sHg}tD]}|jdvr(||jdq(t||rfdd|D}t j dd}t ||st jgtjdWdWdStt |}t d d|D}t ||WdWdSWdn1s0YWdn1s20YdS) aAdds ops to list the names of uninitialized variables. When run, it returns a 1-D tensor containing the names of uninitialized variables if there are any, or an empty array if there are none. Args: var_list: List of `Variable` objects to check. Defaults to the value of `global_variables() + local_variables()` name: Optional name of the `Operation`. Returns: A 1-D tensor containing names of the uninitialized variables, or an empty 1-D tensor if there are no variables or no uninitialized variables. NrrcSsg|]}t|qSr"rrr"r"r#rp rUz2report_uninitialized_variables..Z.TF_DEVICE_FOR_UNINITIALIZED_VARIABLE_REPORTINGz/cpu:0)r[cSsg|] }|jjqSr")rrO)rrr"r"r#r} rU)rrrrbrrrrzr&osenvironrrrr constantrstringr logical_notr boolean_mask)rrOrZ init_varsZ local_deviceZvariables_maskZvariable_names_tensorr"r"r#rS s*    , )N)N)N)N)N)N)r)r)N)Nr)Zr5enumrrvrtensorflow.core.frameworkrrtensorflow.pythonrtensorflow.python.eagerrtensorflow.python.frameworkrrrr tensorflow.python.opsr r r r rrrtensorflow.python.platformrrbtensorflow.python.trackablerrtensorflow.python.typesrtensorflow.python.utilrrrrr"tensorflow.python.util.deprecationrr tensorflow.python.util.tf_exportrr$r%r-Enumr.r;r9rPrrQ Trackablerjr RegisterTyperirrr5rangersrtrurorpryrr#register_tensor_conversion_functionrrrrrrrrrshould_use_resultrrrrrrrrr"r"r"r#s                            " Y +  S%$3Z  "                      ) .