OSError: Not enough disk space. Needed: Unknown size (download: Unknown size, generated: Unknown size, post-processed: Unknown size)

허깅페이스에 데이터셋을 업로드하려고 하니 아래와 같은 에러 발생

disk space가 충분하지 않다고 함

OSError: Not enough disk space. Needed: Unknown size (download: Unknown size, generated: Unknown size, post-processed: Unknown size)

 

has_sufficient_disk_space를 True로 반환하도록 설정하니 해결

from datasets import load_dataset

datasets.builder.has_sufficient_disk_space = lambda needed_bytes, directory=".": True