tutorials 06

ctx=ctx)

  File "O:\PycharmProjects\catdogtf2.2\venv\lib\site-packages\tensorflow\python\eager\execute.py", line 60, in quick_execute

    inputs, attrs, num_outputs)

tensorflow.python.framework.errors_impl.InternalError:  Blas GEMM launch failed : a.shape=(32, 784), b.shape=(784, 512), m=32, n=512, k=784

            [[node sequential/dense/MatMul (defined at O:/PycharmProjects/catdogtf2.2/006.py:51) ]] [Op:__inference_train_function_589]

Function call stack:

train_function

 

O:\PycharmProjects\catdogtf2.2\venv\Scripts\python.exe O:\PyCharm\plugins\python\helpers\pydev\pydevconsole.py --mode=client --port=54455

import sys; print('Python %s on %s' % (sys.version, sys.platform))

sys.path.extend(['O:\\PycharmProjects\\catdogtf2.2', 'O:/PycharmProjects/catdogtf2.2'])

PyDev console: starting.

Python 3.7.7 (default, May  6 2020, 11:45:54) [MSC v.1916 64 bit (AMD64)] on win32

>>> runfile('O:/PycharmProjects/catdogtf2.2/006.py', wdir='O:/PycharmProjects/catdogtf2.2')

2020-08-11 05:29:58.544667: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll

2.2.0

2020-08-11 05:30:01.282588: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll

2020-08-11 05:30:01.324897: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:

pciBusID: 0000:09:00.0 name: GeForce RTX 2080 SUPER computeCapability: 7.5

coreClock: 1.845GHz coreCount: 48 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 462.00GiB/s

2020-08-11 05:30:01.325397: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll

2020-08-11 05:30:01.333925: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll

2020-08-11 05:30:01.339713: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_10.dll

2020-08-11 05:30:01.342729: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_10.dll

2020-08-11 05:30:01.350152: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_10.dll

2020-08-11 05:30:01.354987: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_10.dll

2020-08-11 05:30:01.382853: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll

2020-08-11 05:30:01.383178: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0

2020-08-11 05:30:01.383630: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2

2020-08-11 05:30:01.397994: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1eabe93ddf0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:

2020-08-11 05:30:01.398331: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version

2020-08-11 05:30:01.398700: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:

pciBusID: 0000:09:00.0 name: GeForce RTX 2080 SUPER computeCapability: 7.5

coreClock: 1.845GHz coreCount: 48 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 462.00GiB/s

2020-08-11 05:30:01.399211: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll

2020-08-11 05:30:01.399492: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll

2020-08-11 05:30:01.399779: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_10.dll

2020-08-11 05:30:01.400141: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_10.dll

2020-08-11 05:30:01.400451: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_10.dll

2020-08-11 05:30:01.400725: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_10.dll

2020-08-11 05:30:01.401037: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll

2020-08-11 05:30:01.401408: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0

2020-08-11 05:30:02.056830: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:

2020-08-11 05:30:02.057100: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0

2020-08-11 05:30:02.057241: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N

2020-08-11 05:30:02.057523: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6198 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 SUPER, pci bus id: 0000:09:00.0, compute capability: 7.5)

2020-08-11 05:30:02.061005: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1ea874059e0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:

2020-08-11 05:30:02.061296: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce RTX 2080 SUPER, Compute Capability 7.5

Model: "sequential"

_________________________________________________________________

Layer (type)                 Output Shape              Param #  

=================================================================

dense (Dense)                (None, 512)               401920   

_________________________________________________________________

dropout (Dropout)            (None, 512)               0        

_________________________________________________________________

dense_1 (Dense)              (None, 10)                5130     

=================================================================

Total params: 407,050

Trainable params: 407,050

Non-trainable params: 0

_________________________________________________________________

Epoch 1/10

2020-08-11 05:30:02.840239: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll

28/32 [=========================>....] - ETA: 0s - loss: 1.2189 - accuracy: 0.6618   

Epoch 00001: saving model to training_1/cp.ckpt

32/32 [==============================] - 0s 13ms/step - loss: 1.1645 - accuracy: 0.6750 - val_loss: 0.7062 - val_accuracy: 0.7760

Epoch 2/10

24/32 [=====================>........] - ETA: 0s - loss: 0.4462 - accuracy: 0.8750

Epoch 00002: saving model to training_1/cp.ckpt

32/32 [==============================] - 0s 7ms/step - loss: 0.4426 - accuracy: 0.8740 - val_loss: 0.5803 - val_accuracy: 0.8140

Epoch 3/10

24/32 [=====================>........] - ETA: 0s - loss: 0.2971 - accuracy: 0.9180

Epoch 00003: saving model to training_1/cp.ckpt

32/32 [==============================] - 0s 7ms/step - loss: 0.2959 - accuracy: 0.9170 - val_loss: 0.4929 - val_accuracy: 0.8440

Epoch 4/10

22/32 [===================>..........] - ETA: 0s - loss: 0.2194 - accuracy: 0.9489

Epoch 00004: saving model to training_1/cp.ckpt

32/32 [==============================] - 0s 8ms/step - loss: 0.2136 - accuracy: 0.9510 - val_loss: 0.4599 - val_accuracy: 0.8460

Epoch 5/10

22/32 [===================>..........] - ETA: 0s - loss: 0.1591 - accuracy: 0.9631

Epoch 00005: saving model to training_1/cp.ckpt

32/32 [==============================] - 0s 7ms/step - loss: 0.1630 - accuracy: 0.9630 - val_loss: 0.4229 - val_accuracy: 0.8590

Epoch 6/10

25/32 [======================>.......] - ETA: 0s - loss: 0.1330 - accuracy: 0.9700

Epoch 00006: saving model to training_1/cp.ckpt

32/32 [==============================] - 0s 7ms/step - loss: 0.1269 - accuracy: 0.9710 - val_loss: 0.4153 - val_accuracy: 0.8710

Epoch 7/10

26/32 [=======================>......] - ETA: 0s - loss: 0.0856 - accuracy: 0.9844

Epoch 00007: saving model to training_1/cp.ckpt

32/32 [==============================] - 0s 7ms/step - loss: 0.0904 - accuracy: 0.9840 - val_loss: 0.4451 - val_accuracy: 0.8570

Epoch 8/10

23/32 [====================>.........] - ETA: 0s - loss: 0.0675 - accuracy: 0.9959

Epoch 00008: saving model to training_1/cp.ckpt

32/32 [==============================] - 0s 7ms/step - loss: 0.0716 - accuracy: 0.9950 - val_loss: 0.4232 - val_accuracy: 0.8630

Epoch 9/10

25/32 [======================>.......] - ETA: 0s - loss: 0.0572 - accuracy: 0.9975

Epoch 00009: saving model to training_1/cp.ckpt

32/32 [==============================] - 0s 7ms/step - loss: 0.0555 - accuracy: 0.9980 - val_loss: 0.4201 - val_accuracy: 0.8640

Epoch 10/10

24/32 [=====================>........] - ETA: 0s - loss: 0.0416 - accuracy: 1.0000

Epoch 00010: saving model to training_1/cp.ckpt

32/32 [==============================] - 0s 7ms/step - loss: 0.0432 - accuracy: 0.9980 - val_loss: 0.4192 - val_accuracy: 0.8590

32/32 - 0s - loss: 2.3390 - accuracy: 0.1360

훈련되지 않은 모델의 정확도: 13.60%

32/32 - 0s - loss: 0.4192 - accuracy: 0.8590

복원된 모델의 정확도: 85.90%

WARNING:tensorflow:`period` argument is deprecated. Please use `save_freq` to specify the frequency in number of batches seen.

WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter

WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1

WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2

WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay

WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate

WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.

Epoch 00005: saving model to training_2/cp-0005.ckpt

Epoch 00010: saving model to training_2/cp-0010.ckpt

Epoch 00015: saving model to training_2/cp-0015.ckpt

Epoch 00020: saving model to training_2/cp-0020.ckpt

Epoch 00025: saving model to training_2/cp-0025.ckpt

Epoch 00030: saving model to training_2/cp-0030.ckpt

Epoch 00035: saving model to training_2/cp-0035.ckpt

Epoch 00040: saving model to training_2/cp-0040.ckpt

Epoch 00045: saving model to training_2/cp-0045.ckpt

Epoch 00050: saving model to training_2/cp-0050.ckpt

32/32 - 0s - loss: 0.4963 - accuracy: 0.8730

복원된 모델의 정확도: 87.30%

WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter

WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1

WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2

WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay

WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate

WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.

32/32 - 0s - loss: 0.4963 - accuracy: 0.8730

복원된 모델의 정확도: 87.30%

Epoch 1/5

32/32 [==============================] - 0s 2ms/step - loss: 1.1610 - accuracy: 0.6710

Epoch 2/5

32/32 [==============================] - 0s 3ms/step - loss: 0.4298 - accuracy: 0.8760

Epoch 3/5

32/32 [==============================] - 0s 2ms/step - loss: 0.2997 - accuracy: 0.9150

Epoch 4/5

32/32 [==============================] - 0s 2ms/step - loss: 0.2052 - accuracy: 0.9560

Epoch 5/5

32/32 [==============================] - 0s 2ms/step - loss: 0.1526 - accuracy: 0.9670

2020-08-11 05:30:14.908899: W tensorflow/python/util/util.cc:329] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.

WARNING:tensorflow:From O:\PycharmProjects\catdogtf2.2\venv\lib\site-packages\tensorflow\python\ops\resource_variable_ops.py:1817: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.

Instructions for updating:

If using Keras pass *_constraint arguments to layers.

Model: "sequential_5"

_________________________________________________________________

Layer (type)                 Output Shape              Param #  

=================================================================

dense_10 (Dense)             (None, 512)               401920   

_________________________________________________________________

dropout_5 (Dropout)          (None, 512)               0         

_________________________________________________________________

dense_11 (Dense)             (None, 10)                5130     

=================================================================

Total params: 407,050

Trainable params: 407,050

Non-trainable params: 0

_________________________________________________________________

32/32 - 0s - loss: 0.4585 - accuracy: 0.8460

복원된 모델의 정확도: 84.60%

(1000, 10)

Epoch 1/5

32/32 [==============================] - 0s 2ms/step - loss: 1.1752 - accuracy: 0.6670

Epoch 2/5

32/32 [==============================] - 0s 2ms/step - loss: 0.4129 - accuracy: 0.8900

Epoch 3/5

32/32 [==============================] - 0s 2ms/step - loss: 0.2810 - accuracy: 0.9220

Epoch 4/5

32/32 [==============================] - 0s 2ms/step - loss: 0.2027 - accuracy: 0.9490

Epoch 5/5

32/32 [==============================] - 0s 2ms/step - loss: 0.1418 - accuracy: 0.9770

Model: "sequential_6"

_________________________________________________________________

Layer (type)                 Output Shape              Param #  

=================================================================

dense_12 (Dense)             (None, 512)               401920   

_________________________________________________________________

dropout_6 (Dropout)          (None, 512)               0        

_________________________________________________________________

dense_13 (Dense)             (None, 10)                5130     

=================================================================

Total params: 407,050

Trainable params: 407,050

Non-trainable params: 0

_________________________________________________________________

32/32 - 0s - loss: 0.4315 - accuracy: 0.8680

복원된 모델의 정확도: 86.80%

 

#pip install -q pyyaml h5py  # HDF5 포맷으로 모델을 저장하기 위해서 필요합니다

 

import os

 

import tensorflow as tf

from tensorflow import keras

 

print(tf.version.VERSION)

 

(train_images, train_labels), (test_images, test_labels) = tf.keras.datasets.mnist.load_data()

 

train_labels = train_labels[:1000]

test_labels = test_labels[:1000]

 

train_images = train_images[:1000].reshape(-1, 28 * 28) / 255.0

test_images = test_images[:1000].reshape(-1, 28 * 28) / 255.0

 

# 간단한 Sequential 모델을 정의합니다

def create_model():

  model = tf.keras.models.Sequential([

    keras.layers.Dense(512, activation='relu', input_shape=(784,)),

    keras.layers.Dropout(0.2),

    keras.layers.Dense(10)

  ])

 

  model.compile(optimizer='adam',

                loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),

                metrics=['accuracy'])

 

  return model

 

# 모델 객체를 만듭니다

model = create_model()

 

# 모델 구조를 출력합니다

model.summary()

 

checkpoint_path = "training_1/cp.ckpt"

checkpoint_dir = os.path.dirname(checkpoint_path)

 

# 모델의 가중치를 저장하는 콜백 만들기

cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_path,

                                                 save_weights_only=True,

                                                 verbose=1)

 

# 새로운 콜백으로 모델 훈련하기

model.fit(train_images,

          train_labels,

          epochs=10,

          validation_data=(test_images,test_labels),

          callbacks=[cp_callback])  # 콜백을 훈련에 전달합니다

 

# 옵티마이저의 상태를 저장하는 것과 관련되어 경고가 발생할 수 있습니다.

# 이 경고는 (그리고 이 노트북의 다른 비슷한 경고는) 이전 사용 방식을 권장하지 않기 위함이며 무시해도 좋습니다.

 

#ls {checkpoint_dir}

 

# 기본 모델 객체를 만듭니다

model = create_model()

 

# 모델을 평가합니다

loss, acc = model.evaluate(test_images,  test_labels, verbose=2)

print("훈련되지 않은 모델의 정확도: {:5.2f}%".format(100*acc))

 

# 가중치 로드

model.load_weights(checkpoint_path)

 

# 모델 재평가

loss,acc = model.evaluate(test_images,  test_labels, verbose=2)

print("복원된 모델의 정확도: {:5.2f}%".format(100*acc))

 

# 파일 이름에 에포크 번호를 포함시킵니다(`str.format` 포맷)

checkpoint_path = "training_2/cp-{epoch:04d}.ckpt"

checkpoint_dir = os.path.dirname(checkpoint_path)

 

# 다섯 번째 에포크마다 가중치를 저장하기 위한 콜백을 만듭니다

cp_callback = tf.keras.callbacks.ModelCheckpoint(

    filepath=checkpoint_path,

    verbose=1,

    save_weights_only=True,

    period=5)

 

# 새로운 모델 객체를 만듭니다

model = create_model()

 

# `checkpoint_path` 포맷을 사용하는 가중치를 저장합니다

model.save_weights(checkpoint_path.format(epoch=0))

 

# 새로운 콜백을 사용하여 모델을 훈련합니다

model.fit(train_images,

          train_labels,

          epochs=50,

          callbacks=[cp_callback],

          validation_data=(test_images,test_labels),

          verbose=0)

 

#ls {checkpoint_dir}

 

latest = tf.train.latest_checkpoint(checkpoint_dir)

latest

 

# 새로운 모델 객체를 만듭니다

model = create_model()

 

# 이전에 저장한 가중치를 로드합니다

model.load_weights(latest)

 

# 모델을 재평가합니다

loss, acc = model.evaluate(test_images,  test_labels, verbose=2)

print("복원된 모델의 정확도: {:5.2f}%".format(100*acc))

 

# 가중치를 저장합니다

model.save_weights('./checkpoints/my_checkpoint')

 

# 새로운 모델 객체를 만듭니다

model = create_model()

 

# 가중치를 복원합니다

model.load_weights('./checkpoints/my_checkpoint')

 

# 모델을 평가합니다

loss,acc = model.evaluate(test_images,  test_labels, verbose=2)

print("복원된 모델의 정확도: {:5.2f}%".format(100*acc))

 

# 새로운 모델 객체를 만들고 훈련합니다

model = create_model()

model.fit(train_images, train_labels, epochs=5)

 

# SavedModel로 전체 모델을 저장합니다

#!mkdir -p saved_model

model.save('saved_model/my_model')

 

# my_model 디렉토리

#!ls saved_model

 

# assests 폴더, saved_model.pb, variables 폴더

#!ls saved_model/my_model

 

new_model = tf.keras.models.load_model('saved_model/my_model')

 

# 모델 구조를 확인합니다

new_model.summary()

 

# 복원된 모델을 평가합니다

loss, acc = new_model.evaluate(test_images,  test_labels, verbose=2)

print('복원된 모델의 정확도: {:5.2f}%'.format(100*acc))

 

print(new_model.predict(test_images).shape)

 

# 새로운 모델 객체를 만들고 훈련합니다

model = create_model()

model.fit(train_images, train_labels, epochs=5)

 

# 전체 모델을 HDF5 파일로 저장합니다

# '.h5' 확장자는 이 모델이 HDF5로 저장되었다는 것을 나타냅니다

model.save('my_model.h5')

 

# 가중치와 옵티마이저를 포함하여 정확히 동일한 모델을 다시 생성합니다

new_model = tf.keras.models.load_model('my_model.h5')

 

# 모델 구조를 출력합니다

new_model.summary()

 

loss, acc = new_model.evaluate(test_images,  test_labels, verbose=2)

print('복원된 모델의 정확도: {:5.2f}%'.format(100*acc))

 

 

#

# Copyright (c) 2017 François Chollet

#

# Permission is hereby granted, free of charge, to any person obtaining a

# copy of this software and associated documentation files (the "Software"),

# to deal in the Software without restriction, including without limitation

# the rights to use, copy, modify, merge, publish, distribute, sublicense,

# and/or sell copies of the Software, and to permit persons to whom the

# Software is furnished to do so, subject to the following conditions:

#

# The above copyright notice and this permission notice shall be included in

# all copies or substantial portions of the Software.

#

# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL

# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING

# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER

# DEALINGS IN THE SOFTWARE.