2.2 圖像生成

• 本項(xiàng)目使用 stability.ai 的 stable-diffusion-xl-1024-v1-0 模型生成圖像，最小用例如下

  import os
  import io
  import warnings
  from PIL import Image
  from stability_sdk import client
  import stability_sdk.interfaces.gooseai.generation.generation_pb2 as generation# Our Host URL should not be prepended with "https" nor should it have a trailing slash.
  os.environ['STABILITY_HOST'] = 'grpc.stability.ai:443'# Sign up for an account at the following link to get an API Key.
  # https://platform.stability.ai/# Click on the following link once you have created an account to be taken to your API Key.
  # https://platform.stability.ai/account/keys# Paste your API Key below.# Set up our connection to the API.
  stability_api = client.StabilityInference(key='XXX', # 填入你的 apiverbose=True, # Print debug messages.engine="stable-diffusion-xl-1024-v1-0", # Set the engine to use for generation.# Check out the following link for a list of available engines: https://platform.stability.ai/docs/features/api-parameters#engine
  )# Set up our initial generation parameters.
  answers = stability_api.generate(prompt="expansive landscape rolling greens with gargantuan yggdrasil, intricate world-spanning roots towering under a blue alien sky, masterful, ghibli",seed=4253978046, # If a seed is provided, the resulting generated image will be deterministic.# What this means is that as long as all generation parameters remain the same, you can always recall the same image simply by generating it again.# Note: This isn't quite the case for Clip Guided generations, which we'll tackle in a future example notebook.steps=50, # Amount of inference steps performed on image generation. Defaults to 30. cfg_scale=8.0, # Influences how strongly your generation is guided to match your prompt.# Setting this value higher increases the strength in which it tries to match your prompt.# Defaults to 7.0 if not specified.width=1024, # Generation width, defaults to 512 if not included.height=1024, # Generation height, defaults to 512 if not included.samples=1, # Number of images to generate, defaults to 1 if not included.sampler=generation.SAMPLER_K_DPMPP_2M # Choose which sampler we want to denoise our generation with.# Defaults to k_dpmpp_2m if not specified. Clip Guidance only supports ancestral samplers.# (Available Samplers: ddim, plms, k_euler, k_euler_ancestral, k_heun, k_dpm_2, k_dpm_2_ancestral, k_dpmpp_2s_ancestral, k_lms, k_dpmpp_2m, k_dpmpp_sde)
  )# Set up our warning to print to the console if the adult content classifier is tripped.
  # If adult content classifier is not tripped, save generated images.
  for resp in answers:for artifact in resp.artifacts:if artifact.finish_reason == generation.FILTER:warnings.warn("Your request activated the API's safety filters and could not be processed.""Please modify the prompt and try again.")if artifact.type == generation.ARTIFACT_IMAGE:img = Image.open(io.BytesIO(artifact.binary))img.save(str(artifact.seed)+ ".png") # Save our generated images with their seed number as the filename.
  

• 注意幾點(diǎn)

  1. 本項(xiàng)目當(dāng)前（2023.11.29）使用的包版本為 stability-sdk 0.4.0，文檔參考這里
  2. 這個模型是一個 text-to-image 的模型，生成圖像質(zhì)量會顯著受到 prompt 質(zhì)量影響，因此不適合直接用自然語言作為 prompt 來生成圖像。本 bot 利用 GPT 模型的 in-context learning 能力，先把自然語言翻譯成較高質(zhì)量的 image prompt，再調(diào)用該模型生成圖像，這一步輸入給 GPT 模型的 prompt 如下

     IMGPROMPT = "A prompt example for 一個童話般的寧靜小鎮(zhèn)，鳥瞰視角，動漫風(fēng)格 is “a painting of a fairy tale town, serene landscape, a bird's eye view, anime style, Highly detailed, Vivid Colors.” "
     IMGPROMPT += "Another prompt example for 雙馬尾動漫少女，藍(lán)黑色頭發(fā)，顏色鮮艷 is “a painting of 1girl, blue | black hair, low twintails, anime style, with bright colors, Highly detailed.” "
     IMGPROMPT += "Another prompt example for 擬人化的兔子肖像，油畫，史詩電影風(fēng)格 is “a oil portrait of the bunny, Octane rendering, anthropomorphic creature, reddit moderator, epic, cinematic, elegant, highly detailed, featured on artstation.” "
     IMGPROMPT += "Another prompt example for 黃昏下，大雨中，兩個持刀的海盜在海盜船上決斗 is “Two knife-wielding pirates dueling on a pirate ship, dusk, heavy rain, unreal engine, 8k, high-definition, by Alphonse Mucha and Wayne Barlowe.” "
     IMGPROMPT += "Now write a prompts for "
     

     當(dāng)然，bot 也提供了直接使用用戶輸入內(nèi)容作為 prompt 生成圖像的命令，熟悉 AI 圖像生成方法的用戶可以直接提供高質(zhì)量的 image prompt 序列

2.3 語音輸入 & 輸出

• 本項(xiàng)目使用 OpenAI tts-1 模型實(shí)現(xiàn)文字轉(zhuǎn)語音，使用 whisper-1 模型實(shí)現(xiàn)語音轉(zhuǎn)文字，最小用例如下

  from pathlib import Path
  from openai import OpenAI
  client = OpenAI(api_key='XXX')	# 填入你的 api# text2voice
  speech_file_path = Path(__file__).parent / "speech.ogg"
  response = client.audio.speech.create(model="tts-1",voice="alloy",input="Hello, World! 你好世界！",response_format='opus'
  )
  response.stream_to_file(speech_file_path)# voice2text
  file_path = Path(__file__).parent / "speech.ogg"
  audio_file = open(file_path, "rb")
  transcript = client.audio.transcriptions.create(model="whisper-1", file=audio_file, response_format="text"
  )
  print(transcript)
  

• 本項(xiàng)目使當(dāng)前（2023.11.29）使用的包版本為 openai 1.3.1，文檔參考
  • Text-to-voice: tts-1
  • Voice-to-text: whisper-1