o h2@sddlZddlZddlmZddlmZmZmZmZm Z ddl m Z ddl m Z dede eeed ffd efd d Zded efd dZde eefd eeeeeefffddZ    d$de eefdeeeefdee eeed ffdeeegefded eeeeff ddZGddde ZdZde eefd e j fddZde eefd efddZde eefd efd d!ZGd"d#d#eZdS)%N)Path)AnyCallablecastOptionalUnion)Image) VisionDatasetfilename extensions.returncCs"|t|tr |St|S)aChecks if a file is an allowed extension. Args: filename (string): path to a file extensions (tuple of strings): extensions to consider (lowercase) Returns: bool: True if the filename ends with one of given extensions )lowerendswith isinstancestrtuple)r r ra/home/www/facesmatcher.com/frenv_anti/lib/python3.10/site-packages/torchvision/datasets/folder.pyhas_file_allowed_extension s" rcCs t|tS)zChecks if a file is an allowed image extension. Args: filename (string): path to a file Returns: bool: True if the filename ends with a known image extension )rIMG_EXTENSIONS)r rrr is_image_files r directorycCsFtddt|D}|std|dddt|D}||fS)zWFinds the class folders in a dataset. See :class:`DatasetFolder` for details. css|] }|r|jVqdSN)is_dirname).0entryrrr )szfind_classes..z"Couldn't find any class folder in .cSsi|]\}}||qSrr)ricls_namerrr -sz find_classes..)sortedosscandirFileNotFoundError enumerate)rclasses class_to_idxrrr find_classes$s r*Fr) is_valid_file allow_emptycstj|}|durt|\}}n|stdduo|du}duo&|du}|s+|r/tddur>dtdtffdd }tttgtf|}g}t } t | D]G} || } tj || } tj | sgqSt tj| dd D])\} }}t |D]}tj | |}||r|| f}||| | vr| | qyqpqSt | | }|r|sd d t |d }dur|d ttrnd 7}t||S)zGenerates a list of samples of a form (path_to_sample, class). See :class:`DatasetFolder` for details. Note: The class_to_idx parameter is here optional and will use the logic of the ``find_classes`` function by default. NzE'class_to_index' must have at least one entry to collect any samples.zMBoth extensions and is_valid_file cannot be None or not None at the same timexr cs t|Sr)r)r-r rrr+M z#make_dataset..is_valid_fileT) followlinksz$Found no valid file for the classes z, z. zSupported extensions are: )r$path expanduserr* ValueErrorrboolrrsetr#keysjoinisdirwalkappendaddrr&)rr)r r+r,_Z both_noneZboth_somethingZ instancesZavailable_classesZ target_classZ class_index target_dirrootfnamesfnamer1itemZ empty_classesmsgrr.r make_dataset1sJ      "rCcs<eZdZdZ     ddeeefdeegefde e edfde ed e ed e eege fd e d dffd d Z e   ddeeefdeeefde e edfd e eege fd e d ee eeff ddZdeeefd e eeeeefffddZded e eeffddZd efddZZS) DatasetFolderaA generic data loader. This default directory structure can be customized by overriding the :meth:`find_classes` method. Args: root (str or ``pathlib.Path``): Root directory path. loader (callable): A function to load a sample given its path. extensions (tuple[string]): A list of allowed extensions. both extensions and is_valid_file should not be passed. transform (callable, optional): A function/transform that takes in a sample and returns a transformed version. E.g, ``transforms.RandomCrop`` for images. target_transform (callable, optional): A function/transform that takes in the target and transforms it. is_valid_file (callable, optional): A function that takes path of a file and check if the file is a valid file (used to check of corrupt files) both extensions and is_valid_file should not be passed. allow_empty(bool, optional): If True, empty folders are considered to be valid classes. An error is raised on empty folders if False (default). Attributes: classes (list): List of the class names sorted alphabetically. class_to_idx (dict): Dict with items (class_name, class_index). samples (list): List of (sample path, class_index) tuples targets (list): The class_index value for each image in the dataset NFr>loaderr . transformtarget_transformr+r,r c sjtj|||d||j\}} |j|j| |||d} ||_||_||_| |_| |_ dd| D|_ dS)N)rFrG)r)r r+r,cSsg|]}|dqS)r r)rsrrr sz*DatasetFolder.__init__..) super__init__r*r>rCrEr r(r)samplestargets) selfr>rEr rFrGr+r,r(r)rL __class__rrrKs zDatasetFolder.__init__rr)cCs"|durtdt|||||dS)aGenerates a list of samples of a form (path_to_sample, class). This can be overridden to e.g. read files from a compressed zip file instead of from the disk. Args: directory (str): root dataset directory, corresponding to ``self.root``. class_to_idx (Dict[str, int]): Dictionary mapping class name to class index. extensions (optional): A list of allowed extensions. Either extensions or is_valid_file should be passed. Defaults to None. is_valid_file (optional): A function that takes path of a file and checks if the file is a valid file (used to check of corrupt files) both extensions and is_valid_file should not be passed. Defaults to None. allow_empty(bool, optional): If True, empty folders are considered to be valid classes. An error is raised on empty folders if False (default). Raises: ValueError: In case ``class_to_idx`` is empty. ValueError: In case ``extensions`` and ``is_valid_file`` are None or both are not None. FileNotFoundError: In case no valid file was found for any class. Returns: List[Tuple[str, int]]: samples of a form (path_to_sample, class) Nz*The class_to_idx parameter cannot be None.)r r+r,)r3rC)rr)r r+r,rrrrCs   zDatasetFolder.make_datasetcCst|S)utFind the class folders in a dataset structured as follows:: directory/ ├── class_x │ ├── xxx.ext │ ├── xxy.ext │ └── ... │ └── xxz.ext └── class_y ├── 123.ext ├── nsdf3.ext └── ... └── asd932_.ext This method can be overridden to only consider a subset of classes, or to adapt to a different dataset directory structure. Args: directory(str): Root directory path, corresponding to ``self.root`` Raises: FileNotFoundError: If ``dir`` has no class folders. Returns: (Tuple[List[str], Dict[str, int]]): List of all classes and dictionary mapping each class to an index. )r*)rNrrrrr*szDatasetFolder.find_classesindexcCsH|j|\}}||}|jdur||}|jdur ||}||fS)z Args: index (int): Index Returns: tuple: (sample, target) where target is class_index of the target class. N)rLrErFrG)rNrQr1targetsamplerrr __getitem__s     zDatasetFolder.__getitem__cCs t|jSr)lenrL)rNrrr__len__r/zDatasetFolder.__len__)NNNNF)NNF)__name__ __module__ __qualname____doc__rrrrrrrr4rK staticmethoddictintlistrCr*rTrV __classcell__rrrOrrDmsX       .(rD) z.jpgz.jpegz.pngz.ppmz.bmpz.pgmz.tifz.tiffz.webpr1cCsBt|d}t|}|dWdS1swYdS)NrbRGB)openrconvert)r1fZimgrrr pil_loaders  $recCs0ddl}z||WStyt|YSw)Nr)accimagerOSErrorre)r1rfrrraccimage_loader s    rhcCs&ddlm}|dkrt|St|S)Nr)get_image_backendrf)Z torchvisionrirhre)r1rirrrdefault_loaders  rjcsneZdZdZddeddfdeeefdee dee de ege fdee ege fd e f fd d Z Z S) ImageFolderaA generic data loader where the images are arranged in this way by default: :: root/dog/xxx.png root/dog/xxy.png root/dog/[...]/xxz.png root/cat/123.png root/cat/nsdf3.png root/cat/[...]/asd932_.png This class inherits from :class:`~torchvision.datasets.DatasetFolder` so the same methods can be overridden to customize the dataset. Args: root (str or ``pathlib.Path``): Root directory path. transform (callable, optional): A function/transform that takes in a PIL image or torch.Tensor, depends on the given loader, and returns a transformed version. E.g, ``transforms.RandomCrop`` target_transform (callable, optional): A function/transform that takes in the target and transforms it. loader (callable, optional): A function to load an image given its path. is_valid_file (callable, optional): A function that takes path of an Image file and check if the file is a valid file (used to check of corrupt files) allow_empty(bool, optional): If True, empty folders are considered to be valid classes. An error is raised on empty folders if False (default). Attributes: classes (list): List of the class names sorted alphabetically. class_to_idx (dict): Dict with items (class_name, class_index). imgs (list): List of (image path, class_index) tuples NFr>rFrGrEr+r,c s2tj|||dur tnd||||d|j|_dS)N)rFrGr+r,)rJrKrrLZimgs)rNr>rFrGrEr+r,rOrrrK?s  zImageFolder.__init__)rWrXrYrZrjrrrrrrr4rKr_rrrOrrks("  rk)NNNF)r$Zos.pathpathlibrtypingrrrrrZPILrZvisionr rrr4rrr^r\r]r*rCrDrrerhrjrkrrrrsB   & .   <