학습 테스트 베드(윈도우)

암 종류별 사진 크롤링 이후에는

비슷한 것 끼리 묶고

묶여진 자료만 학습

 

테스트베드는 윈도우 데탑... 다른 애들은 다들 일해서...

 

C:\Users\joe>pip3 install tensorflow

Collecting tensorflow

  Downloading https://files.pythonhosted.org/packages/af/50/d7da24189d95e2084bb1cc350a8e4acdf1b0c9b3d57def7a348f0d9cb062/tensorflow-2.2.0-cp37-cp37m-win_amd64.whl (459.2MB)

     |█                               | 15.7MB 204kB/s eta 0:36:06    

설치

엥 그런데 파이썬 어딨었지?

뭐가 진짜야

C:\Users\joe>echo %PATH%

o:\WinAVR\bin;o:\WinAVR\utils\bin;O:\arm\DS-5 v5.27.1\sw\ARMCompiler5.06u5\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;J:\!!!!!!!!\visualSVN\bin;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Xoreax\IncrediBuild;C:\Program Files (x86)\Bitvise SSH Client;O:\CMake\bin;o:\Git LFS;C:\Program Files\Git\cmd;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Program Files (x86)\ePapyrus\Papyrus-PlugIn;C:\Program Files (x86)\ePapyrus\Papyrus-PlugIn\Addins;J:\Program Files\Java\jdk-11.0.3\bin;J:\Program Files\Java\jdk-11.0.3\bin;C:\ProgramData\chocolatey\bin;;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Users\joe\AppData\Local\Microsoft\WindowsApps;;o:\PyCharm\bin;

C:\Users\joe\AppData\Local\Microsoft\WindowsApps\python3.exe

 

얘군.

 

진행 중이던 내용은 역시나 에러 발생

 

WARNING: The script f2py.exe is installed in 'C:\Users\joe\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\Scripts' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: 'C:\\Users\\joe\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python37\\site-packages\\tensorflow_estimator\\python\\estimator\\canned\\linear_optimizer\\python\\utils\\__pycache__\\sharded_mutable_dense_hashtable.cpython-37.pyc'

 

WARNING: You are using pip version 19.2.3, however version 20.1.1 is available.

You should consider upgrading via the 'python -m pip install --upgrade pip' command.

 

pip3로 깔았는데 ㅡㅡ;

 

믓튼,

C:\Users\joe>python -m pip install --upgrade pip

Collecting pip

  Downloading https://files.pythonhosted.org/packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl (1.5MB)

     |██████████████                  | 655kB 70kB/s eta 0:00:12

마찬가지겠지.

kdown, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, idna, certifi, urllib3, chardet, requests, oauthlib, requests-oauthlib, google-auth-oauthlib, tensorboard, tensorflow

  WARNING: The script wheel.exe is installed in 'C:\Users\joe\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\Scripts' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: 'C:\\Users\\joe\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python37\\site-packages\\tensorflow_estimator\\python\\estimator\\canned\\linear_optimizer\\python\\utils\\__pycache__\\sharded_mutable_dense_hashtable.cpython-37.pyc'

 

 역시나 

C:\Users\joe>

C:\Users\joe>pip3 install tensorflow-gpu

Collecting tensorflow-gpu

  Downloading tensorflow_gpu-2.2.0-cp37-cp37m-win_amd64.whl (460.4 MB)

     |                                | 256 kB 192 kB/s eta 0:39:46

 

윈도우 환경 텐서면 gpu로 가겠지? 하고 gpu로 재시도.

 

 

sts-oauthlib, google-auth-oauthlib, tensorboard, wrapt, google-pasta, gast, termcolor, scipy, opt-einsum, astunparse, tensorflow-gpu-estimator, tensorflow-gpu

  WARNING: The script chardetect.exe is installed in 'C:\Users\joe\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\Scripts' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  WARNING: The scripts pyrsa-decrypt.exe, pyrsa-encrypt.exe, pyrsa-keygen.exe, pyrsa-priv2pub.exe, pyrsa-sign.exe and pyrsa-verify.exe are installed in 'C:\Users\joe\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\Scripts' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: 'C:\\Users\\joe\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python37\\site-packages\\tensorboard_plugin_wit\\_vendor\\tensorflow_serving\\sources\\storage_path\\__pycache__\\file_system_storage_path_source_pb2.cpython-37.pyc'

 

아나콘다 가야겠네.

 

아나콘다로 모두 설치하고,

파이참으로 마무리.

 

리눅스도 우분투 전에는 늘 소스로 설치해야 하고, 순서나 패키지 궁합이 맞지 않을 때 수 없이 지웠다 재설치를 반복해야 했었다. 이제는 깔끔하게 환경 설정이 되지 않는 경우 해당 시스템을 바꿀 정도라... 뭐, 그래서 지금은 숙성된 AI를 하기 딱 좋은 시기인 듯 보인다. 

 

J:\Anaconda3\python.exe O:/PycharmProjects/test001/bs/tensorflow-gpu_test.py

2020-06-24 01:57:56.286036: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll

2020-06-24 01:58:00.182514: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll

2020-06-24 01:58:00.220162: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:

pciBusID: 0000:09:00.0 name: GeForce RTX 2080 SUPER computeCapability: 7.5

coreClock: 1.845GHz coreCount: 48 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 462.00GiB/s

2020-06-24 01:58:00.220480: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll

2020-06-24 01:58:00.228728: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll

2020-06-24 01:58:00.233755: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_10.dll

2020-06-24 01:58:00.235671: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_10.dll

2020-06-24 01:58:00.240970: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_10.dll

2020-06-24 01:58:00.244173: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_10.dll

2020-06-24 01:58:00.255182: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll

2020-06-24 01:58:00.255874: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0

2020-06-24 01:58:00.256555: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2

2020-06-24 01:58:00.261185: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:

pciBusID: 0000:09:00.0 name: GeForce RTX 2080 SUPER computeCapability: 7.5

coreClock: 1.845GHz coreCount: 48 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 462.00GiB/s

2020-06-24 01:58:00.261506: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll

2020-06-24 01:58:00.261658: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll

2020-06-24 01:58:00.261815: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_10.dll

2020-06-24 01:58:00.261989: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_10.dll

2020-06-24 01:58:00.262142: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_10.dll

2020-06-24 01:58:00.262296: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_10.dll

2020-06-24 01:58:00.262461: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll

2020-06-24 01:58:00.263027: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0

2020-06-24 01:58:00.997862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:

2020-06-24 01:58:00.998035: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102]      0

2020-06-24 01:58:00.998133: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0:   N

2020-06-24 01:58:00.999020: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6265 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 SUPER, pci bus id: 0000:09:00.0, compute capability: 7.5)

2020-06-24 01:58:01.003901: I tensorflow/core/common_runtime/eager/execute.cc:573] Executing op MatMul in device /job:localhost/replica:0/task:0/device:GPU:0

2020-06-24 01:58:01.004173: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll

tf.Tensor(

[[22. 28.]

 [49. 64.]], shape=(2, 2), dtype=float32)

 

소스는

https://www.tensorflow.org/guide/gpu?hl=ko여기 참고

GPU 사용하기  |  TensorFlow Core

# import os
# os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
import tensorflow as tf
tf.debugging.set_log_device_placement(True)
#
# hello = tf.constant('Hello, TensorFlow!')
#
# print(hello.numpy())

# 텐서 생성
a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
c = tf.matmul(a, b)

print(c)

 

그런데 2080 super 8GB 샀던 것 같은데 왜 6GB로 나오는겨 --> 6GB 를 설정하는거네. 2GB니 하던 게임이 죽는다.

https://developer.nvidia.com/cuda-gpus#compute

 

CUDA GPUs

엥 Compute Capability 는 ...

GPUCompute Capability

Quadro RTX 8000

7.5

얘랑 같다고?

ARM Firmware 강사라고 중간에 어떻게 연결되는지 파다가는 인생도 멍들 것 같아서 그러려니 하고 일단 넘어가야 겠다.

음... 15년 게임컴 하다... 나중에 클라우드만 쓰다 다시 개인컴이라 직접 체감하는 수 밖에는... 그래도 2080인데 또 무한 대기는 아니겠지.