了解如何設(shè)置Azure中 JSONL 文件格式，以便在訓(xùn)練和推理期間在計算機(jī)視覺任務(wù)的自動化 ML 實驗中使用數(shù)據(jù)。

關(guān)注TechLead，分享AI全維度知識。作者擁有10+年互聯(lián)網(wǎng)服務(wù)架構(gòu)、AI產(chǎn)品研發(fā)經(jīng)驗、團(tuán)隊管理經(jīng)驗，同濟(jì)本復(fù)旦碩，復(fù)旦機(jī)器人智能實驗室成員，阿里云認(rèn)證的資深架構(gòu)師，項目管理專業(yè)人士，上億營收AI產(chǎn)品研發(fā)負(fù)責(zé)人。

一、用于訓(xùn)練的數(shù)據(jù)架構(gòu)

Azure 機(jī)器學(xué)習(xí)的圖像 AutoML 要求以 JSONL（JSON 行）格式準(zhǔn)備輸入圖像數(shù)據(jù)。 本部分介紹多類圖像分類、多標(biāo)簽圖像分類、對象檢測和實例分段的輸入數(shù)據(jù)格式或架構(gòu)。 我們還將提供最終訓(xùn)練或驗證 JSON 行文件的示例。

圖像分類（二進(jìn)制/多類）

每個 JSON 行中的輸入數(shù)據(jù)格式/架構(gòu)：

{"image_url":"AmlDatastore://data_directory/../Image_name.image_format","image_details":{"format":"image_format","width":"image_width","height":"image_height"},"label":"class_name",
}

密鑰	說明	示例
image_url	Azure 機(jī)器學(xué)習(xí)數(shù)據(jù)存儲中的圖像位置
Required, String	"AmlDatastore://data_directory/Image_01.jpg"
image_details	圖像詳細(xì)信息
Optional, Dictionary	"image_details":{"format": "jpg", "width": "400px", "height": "258px"}
format	圖像類型（支持 Pillow 庫中所有可用的圖像格式）
Optional, String from {"jpg", "jpeg", "png", "jpe", "jfif","bmp", "tif", "tiff"}	"jpg" or "jpeg" or "png" or "jpe" or "jfif" or "bmp" or "tif" or "tiff"
width	圖像的寬度
Optional, String or Positive Integer	"400px" or 400
height	圖像的高度
Optional, String or Positive Integer	"200px" or 200
label	圖像的類/標(biāo)簽
Required, String	"cat"

多類圖像分類的 JSONL 文件示例：

{"image_url": "AmlDatastore://image_data/Image_01.jpg", "image_details":{"format": "jpg", "width": "400px", "height": "258px"}, "label": "can"}
{"image_url": "AmlDatastore://image_data/Image_02.jpg", "image_details": {"format": "jpg", "width": "397px", "height": "296px"}, "label": "milk_bottle"}
.
.
.
{"image_url": "AmlDatastore://image_data/Image_n.jpg", "image_details": {"format": "jpg", "width": "1024px", "height": "768px"}, "label": "water_bottle"}

多標(biāo)簽圖像分類

下面是每個 JSON 行中用于圖像分類的輸入數(shù)據(jù)格式/架構(gòu)示例。

{"image_url":"AmlDatastore://data_directory/../Image_name.image_format","image_details":{"format":"image_format","width":"image_width","height":"image_height"},"label":["class_name_1","class_name_2","class_name_3","...","class_name_n"]
}

密鑰	說明	示例
image_url	Azure 機(jī)器學(xué)習(xí)數(shù)據(jù)存儲中的圖像位置
Required, String	"AmlDatastore://data_directory/Image_01.jpg"
image_details	圖像詳細(xì)信息
Optional, Dictionary	"image_details":{"format": "jpg", "width": "400px", "height": "258px"}
format	圖像類型（支持 Pillow 庫中所有可用的圖像格式）
Optional, String from {"jpg", "jpeg", "png", "jpe", "jfif", "bmp", "tif", "tiff"}	"jpg" or "jpeg" or "png" or "jpe" or "jfif" or "bmp" or "tif" or "tiff"
width	圖像的寬度
Optional, String or Positive Integer	"400px" or 400
height	圖像的高度
Optional, String or Positive Integer	"200px" or 200
label	圖像中的類/標(biāo)簽列表
Required, List of Strings	["cat","dog"]

多標(biāo)簽圖像分類的 JSONL 文件示例：

{"image_url": "AmlDatastore://image_data/Image_01.jpg", "image_details":{"format": "jpg", "width": "400px", "height": "258px"}, "label": ["can"]}
{"image_url": "AmlDatastore://image_data/Image_02.jpg", "image_details": {"format": "jpg", "width": "397px", "height": "296px"}, "label": ["can","milk_bottle"]}
.
.
.
{"image_url": "AmlDatastore://image_data/Image_n.jpg", "image_details": {"format": "jpg", "width": "1024px", "height": "768px"}, "label": ["carton","milk_bottle","water_bottle"]}

對象檢測

下面是用于對象檢測的示例 JSONL 文件。

{"image_url":"AmlDatastore://data_directory/../Image_name.image_format","image_details":{"format":"image_format","width":"image_width","height":"image_height"},"label":[{"label":"class_name_1","topX":"xmin/width","topY":"ymin/height","bottomX":"xmax/width","bottomY":"ymax/height","isCrowd":"isCrowd"},{"label":"class_name_2","topX":"xmin/width","topY":"ymin/height","bottomX":"xmax/width","bottomY":"ymax/height","isCrowd":"isCrowd"},"..."]
}

其中：

• xmin = 邊界框左上角的 x 坐標(biāo)
• ymin = 邊界框左上角的 y 坐標(biāo)
• xmax = 邊界框右下角的 x 坐標(biāo)
• ymax = 邊界框右下角的 y 坐標(biāo)

密鑰	說明	示例
image_url	Azure 機(jī)器學(xué)習(xí)數(shù)據(jù)存儲中的圖像位置
Required, String	"AmlDatastore://data_directory/Image_01.jpg"
image_details	圖像詳細(xì)信息
Optional, Dictionary	"image_details":{"format": "jpg", "width": "400px", "height": "258px"}
format	圖像類型（支持 Pillow 庫中提供的所有圖像格式。但對于 YOLO，僅支持 opencv 允許的圖像格式）
Optional, String from {"jpg", "jpeg", "png", "jpe", "jfif", "bmp", "tif", "tiff"}	"jpg" or "jpeg" or "png" or "jpe" or "jfif" or "bmp" or "tif" or "tiff"
width	圖像的寬度
Optional, String or Positive Integer	"499px" or 499
height	圖像的高度
Optional, String or Positive Integer	"665px" or 665
label（外部鍵）	邊界框列表，其中每個框都是其左上方和右下方坐標(biāo)的 label, topX, topY, bottomX, bottomY, isCrowd 字典
Required, List of dictionaries	[{"label": "cat", "topX": 0.260, "topY": 0.406, "bottomX": 0.735, "bottomY": 0.701, "isCrowd": 0}]
label（內(nèi)部鍵）	邊界框中對象的類/標(biāo)簽
Required, String	"cat"
topX	邊界框左上角的 x 坐標(biāo)與圖像寬度的比率
Required, Float in the range [0,1]	0.260
topY	邊界框左上角的 y 坐標(biāo)與圖像高度的比率
Required, Float in the range [0,1]	0.406
bottomX	邊界框右下角的 x 坐標(biāo)與圖像寬度的比率
Required, Float in the range [0,1]	0.735
bottomY	邊界框右下角的 y 坐標(biāo)與圖像高度的比率
Required, Float in the range [0,1]	0.701
isCrowd	指示邊界框是否圍繞對象群。 如果設(shè)置了此特殊標(biāo)志，我們在計算指標(biāo)時將跳過此特定邊界框。
Optional, Bool	0

用于對象檢測的 JSONL 文件示例：

{"image_url": "AmlDatastore://image_data/Image_01.jpg", "image_details": {"format": "jpg", "width": "499px", "height": "666px"}, "label": [{"label": "can", "topX": 0.260, "topY": 0.406, "bottomX": 0.735, "bottomY": 0.701, "isCrowd": 0}]}
{"image_url": "AmlDatastore://image_data/Image_02.jpg", "image_details": {"format": "jpg", "width": "499px", "height": "666px"}, "label": [{"label": "carton", "topX": 0.172, "topY": 0.153, "bottomX": 0.432, "bottomY": 0.659, "isCrowd": 0}, {"label": "milk_bottle", "topX": 0.300, "topY": 0.566, "bottomX": 0.891, "bottomY": 0.735, "isCrowd": 0}]}
.
.
.
{"image_url": "AmlDatastore://image_data/Image_n.jpg", "image_details": {"format": "jpg", "width": "499px", "height": "666px"}, "label": [{"label": "carton", "topX": 0.0180, "topY": 0.297, "bottomX": 0.380, "bottomY": 0.836, "isCrowd": 0}, {"label": "milk_bottle", "topX": 0.454, "topY": 0.348, "bottomX": 0.613, "bottomY": 0.683, "isCrowd": 0}, {"label": "water_bottle", "topX": 0.667, "topY": 0.279, "bottomX": 0.841, "bottomY": 0.615, "isCrowd": 0}]}

實例分段

對于實例分段，自動化 ML 僅支持多邊形作為輸入和輸出，不支持掩碼。

下面是實例分段的示例 JSONL 文件。

{"image_url":"AmlDatastore://data_directory/../Image_name.image_format","image_details":{"format":"image_format","width":"image_width","height":"image_height"},"label":[{"label":"class_name","isCrowd":"isCrowd","polygon":[["x1", "y1", "x2", "y2", "x3", "y3", "...", "xn", "yn"]]}]
}

密鑰	說明	示例
image_url	Azure 機(jī)器學(xué)習(xí)數(shù)據(jù)存儲中的圖像位置
Required, String	"AmlDatastore://data_directory/Image_01.jpg"
image_details	圖像詳細(xì)信息
Optional, Dictionary	"image_details":{"format": "jpg", "width": "400px", "height": "258px"}
format	映像類型
Optional, String from {"jpg", "jpeg", "png", "jpe", "jfif", "bmp", "tif", "tiff" }	"jpg" or "jpeg" or "png" or "jpe" or "jfif" or "bmp" or "tif" or "tiff"
width	圖像的寬度
Optional, String or Positive Integer	"499px" or 499
height	圖像的高度
Optional, String or Positive Integer	"665px" or 665
label（外部鍵）	掩碼列表，其中每個掩碼都是 label, isCrowd, polygon coordinates 的字典
Required, List of dictionaries	[{"label": "can", "isCrowd": 0, "polygon": [[0.577, 0.689,
0.562, 0.681,
0.559, 0.686]]}]
label（內(nèi)部鍵）	掩碼中對象的類/標(biāo)簽
Required, String	"cat"
isCrowd	指示掩碼是否圍繞對象群
Optional, Bool	0
polygon	對象的多邊形坐標(biāo)
Required, List of list for multiple segments of the same instance. Float values in the range [0,1]	[[0.577, 0.689, 0.567, 0.689, 0.559, 0.686]]

實例分段的 JSONL 文件示例：

{"image_url": "AmlDatastore://image_data/Image_01.jpg", "image_details": {"format": "jpg", "width": "499px", "height": "666px"}, "label": [{"label": "can", "isCrowd": 0, "polygon": [[0.577, 0.689, 0.567, 0.689, 0.559, 0.686, 0.380, 0.593, 0.304, 0.555, 0.294, 0.545, 0.290, 0.534, 0.274, 0.512, 0.2705, 0.496, 0.270, 0.478, 0.284, 0.453, 0.308, 0.432, 0.326, 0.423, 0.356, 0.415, 0.418, 0.417, 0.635, 0.493, 0.683, 0.507, 0.701, 0.518, 0.709, 0.528, 0.713, 0.545, 0.719, 0.554, 0.719, 0.579, 0.713, 0.597, 0.697, 0.621, 0.695, 0.629, 0.631, 0.678, 0.619, 0.683, 0.595, 0.683, 0.577, 0.689]]}]}
{"image_url": "AmlDatastore://image_data/Image_02.jpg", "image_details": {"format": "jpg", "width": "499px", "height": "666px"}, "label": [{"label": "carton", "isCrowd": 0, "polygon": [[0.240, 0.65, 0.234, 0.654, 0.230, 0.647, 0.210, 0.512, 0.202, 0.403, 0.182, 0.267, 0.184, 0.243, 0.180, 0.166, 0.186, 0.159, 0.198, 0.156, 0.396, 0.162, 0.408, 0.169, 0.406, 0.217, 0.414, 0.249, 0.422, 0.262, 0.422, 0.569, 0.342, 0.569, 0.334, 0.572, 0.320, 0.585, 0.308, 0.624, 0.306, 0.648, 0.240, 0.657]]}, {"label": "milk_bottle",  "isCrowd": 0, "polygon": [[0.675, 0.732, 0.635, 0.731, 0.621, 0.725, 0.573, 0.717, 0.516, 0.717, 0.505, 0.720, 0.462, 0.722, 0.438, 0.719, 0.396, 0.719, 0.358, 0.714, 0.334, 0.714, 0.322, 0.711, 0.312, 0.701, 0.306, 0.687, 0.304, 0.663, 0.308, 0.630, 0.320, 0.596, 0.32, 0.588, 0.326, 0.579]]}]}
.
.
.
{"image_url": "AmlDatastore://image_data/Image_n.jpg", "image_details": {"format": "jpg", "width": "499px", "height": "666px"}, "label": [{"label": "water_bottle", "isCrowd": 0, "polygon": [[0.334, 0.626, 0.304, 0.621, 0.254, 0.603, 0.164, 0.605, 0.158, 0.602, 0.146, 0.602, 0.142, 0.608, 0.094, 0.612, 0.084, 0.599, 0.080, 0.585, 0.080, 0.539, 0.082, 0.536, 0.092, 0.533, 0.126, 0.530, 0.132, 0.533, 0.144, 0.533, 0.162, 0.525, 0.172, 0.525, 0.186, 0.521, 0.196, 0.521 ]]}, {"label": "milk_bottle", "isCrowd": 0, "polygon": [[0.392, 0.773, 0.380, 0.732, 0.379, 0.767, 0.367, 0.755, 0.362, 0.735, 0.362, 0.714, 0.352, 0.644, 0.352, 0.611, 0.362, 0.597, 0.40, 0.593, 0.444,  0.494, 0.588, 0.515, 0.585, 0.621, 0.588, 0.671, 0.582, 0.713, 0.572, 0.753 ]]}]}

---

二、用于推理的數(shù)據(jù)格式

在本部分中，我們將記錄在使用部署的模型時進(jìn)行預(yù)測所需的輸入數(shù)據(jù)格式。 可以接受內(nèi)容類型為 application/octet-stream 的任何上述圖像格式。

輸入格式

下面是使用特定于任務(wù)的模型終結(jié)點對任何任務(wù)生成預(yù)測所需的輸入格式。 部署模型后，我們可以使用以下代碼段來獲取所有任務(wù)的預(yù)測。

# input image for inference
sample_image = './test_image.jpg'
# load image data
data = open(sample_image, 'rb').read()
# set the content type
headers = {'Content-Type': 'application/octet-stream'}
# if authentication is enabled, set the authorization header
headers['Authorization'] = f'Bearer {key}'
# make the request and display the response
response = requests.post(scoring_uri, data, headers=headers)

輸出格式

根據(jù)任務(wù)類型，對模型終結(jié)點進(jìn)行的預(yù)測遵循不同的結(jié)構(gòu)。 本部分將探討多類、多標(biāo)簽圖像分類、對象檢測和實例分段任務(wù)的輸出數(shù)據(jù)格式。

圖像分類

圖像分類的終結(jié)點返回數(shù)據(jù)集中的所有標(biāo)簽及其在輸入圖像中的概率分?jǐn)?shù)，格式如下：

{"filename":"/tmp/tmppjr4et28","probs":[2.098e-06,4.783e-08,0.999,8.637e-06],"labels":["can","carton","milk_bottle","water_bottle"]
}

多標(biāo)簽圖像分類

對于多標(biāo)簽圖像分類，模型終結(jié)點返回標(biāo)簽及其概率。

{"filename":"/tmp/tmpsdzxlmlm","probs":[0.997,0.960,0.982,0.025],"labels":["can","carton","milk_bottle","water_bottle"]
}

對象檢測

對象檢測模型返回多個框，其中包含縮放后的左上角和右下角坐標(biāo)，以及框標(biāo)簽和置信度分?jǐn)?shù)。

{"filename":"/tmp/tmpdkg2wkdy","boxes":[{"box":{"topX":0.224,"topY":0.285,"bottomX":0.399,"bottomY":0.620},"label":"milk_bottle","score":0.937},{"box":{"topX":0.664,"topY":0.484,"bottomX":0.959,"bottomY":0.812},"label":"can","score":0.891},{"box":{"topX":0.423,"topY":0.253,"bottomX":0.632,"bottomY":0.725},"label":"water_bottle","score":0.876}]
}

實例分段

在實例分段中，輸出包含多個框，其中包含縮放后的左上角和右下角坐標(biāo)、標(biāo)簽、置信度和多邊形（非掩碼）。 此處，多邊形值與我們在“架構(gòu)”部分中討論的格式相同。

{"filename":"/tmp/tmpi8604s0h","boxes":[{"box":{"topX":0.679,"topY":0.491,"bottomX":0.926,"bottomY":0.810},"label":"can","score":0.992,"polygon":[[0.82, 0.811, 0.771, 0.810, 0.758, 0.805, 0.741, 0.797, 0.735, 0.791, 0.718, 0.785, 0.715, 0.778, 0.706, 0.775, 0.696, 0.758, 0.695, 0.717, 0.698, 0.567, 0.705, 0.552, 0.706, 0.540, 0.725, 0.520, 0.735, 0.505, 0.745, 0.502, 0.755, 0.493]]},{"box":{"topX":0.220,"topY":0.298,"bottomX":0.397,"bottomY":0.601},"label":"milk_bottle","score":0.989,"polygon":[[0.365, 0.602, 0.273, 0.602, 0.26, 0.595, 0.263, 0.588, 0.251, 0.546, 0.248, 0.501, 0.25, 0.485, 0.246, 0.478, 0.245, 0.463, 0.233, 0.442, 0.231, 0.43, 0.226, 0.423, 0.226, 0.408, 0.234, 0.385, 0.241, 0.371, 0.238, 0.345, 0.234, 0.335, 0.233, 0.325, 0.24, 0.305, 0.586, 0.38, 0.592, 0.375, 0.598, 0.365]]},{"box":{"topX":0.433,"topY":0.280,"bottomX":0.621,"bottomY":0.679},"label":"water_bottle","score":0.988,"polygon":[[0.576, 0.680, 0.501, 0.680, 0.475, 0.675, 0.460, 0.625, 0.445, 0.630, 0.443, 0.572, 0.440, 0.560, 0.435, 0.515, 0.431, 0.501, 0.431, 0.433, 0.433, 0.426, 0.445, 0.417, 0.456, 0.407, 0.465, 0.381, 0.468, 0.327, 0.471, 0.318]]}]
}

關(guān)注TechLead，分享AI全維度知識。作者擁有10+年互聯(lián)網(wǎng)服務(wù)架構(gòu)、AI產(chǎn)品研發(fā)經(jīng)驗、團(tuán)隊管理經(jīng)驗，同濟(jì)本復(fù)旦碩，復(fù)旦機(jī)器人智能實驗室成員，阿里云認(rèn)證的資深架構(gòu)師，項目管理專業(yè)人士，上億營收AI產(chǎn)品研發(fā)負(fù)責(zé)人。