Pour cold coffee into an ice cube tray, and freeze until firm. Stir bitters, vanilla extract, and sugar into remaining coffee. Fill tall glasses with frozen coffee cubes. Place a long teaspoon in each glass to prevent glass from crackling if adding hot coffee. Pour in remaining coffee, and serve immediately.