The Imperial Household Agency has made changes and now it is possible for visitors to participate in the tour of the palace without prior arrangements.
Tours will be held twice a day, at 10am and 1:30pm from Tuesdays to Saturdays, and visitors should turn up at Kikyomon Gate at least 30 minutes before the time and have a passport as ID.
See the details on their website:
http://www.kunaicho.go.jp/e-event/toujitsu-sankan-start.html
Also a newspaper piece on the new system:
http://www.japantimes.co.jp/news/2016/06/25/national/imperial-palace-tour-now-available-without-prior-reservation/