Movie reconstruction from human brain activity

The errors and bias in these systems add to the ethical concerns. While it is arguably a good thing to substitute torture with brain image interrogation, there is a risk of putting too much trust in these systems, subjecting innocents to days of leading investigation, just because the computer said there is something to be found there. Israeli airport security (arguably the best in the world) deploys derivatives of these systems, that look at micro-gestures, elevation of heart rates, pupil dilation, and temperature changes, to see if passengers respond with familiarity to terrorist imagery flashed on a screen as they walk by it. If that already works in practice, imagine the same, but being strapped with hundreds of sensors. See also the 2010 research on image reconstruction from brain activity, and extrapolate that 10 years in the future and applied to military interrogation: ▶ this video

in “Machine translation of cortical activity to text with encoder–decoder framework” · View on HN

I feel like we are so close to wirelessly monitoring all brain waves (next 50 years) we will probably be able to, in real time, recognize and diagnose abnormal brain patterns. I think it would be promising if we get this sensor tech up and then perhaps deploy nanotech to interfere with maladaptive brain patterns. The ethics of such a device is considerably sticky. TED talk by Jack Gallant [link] recreating vision: ▶ this video another ted talk by John-Dylan Haynes [link] paraphrased: "wireless non-contact mind reading tech may be cheap and mass producible within the next 50 years." the only rate limiting step with his method of decoding thoughts is that it takes time under a high energy device to pair associations in the brain. Think of how much screen time we currently use. If we could pair a wireless 'headset' like headphones while looking at pictures on a screen it would be possible to build a brain bank of children->tenenagers. Currently we just need a lot of processing power and reserved bandwidth on the internet to do some cool stuff.

in “Evidence of a nocebo response following a nationwide antidepressant drug switch” · View on HN

Reminds me of this project from 2011: ▶ this video If I were a billionaire, one of the projects I'd do would be to apply this sort of technology at scale, upon volunteers who were witness to poorly recorded historic events. WWII, Beatles concerts, JFK assassination, 9/11... all of them have been recorded from angles/perspectives that exist only in peoples' memories, and the longer we wait, the more of those memories will be lost.

in “Deep image reconstruction from human brain activity (2017)” · View on HN

Extremely interesting to compare with Jack Gallant's reconstruction of video from the brain MRI of a viewer: ▶ this video

in “VOX.com – A guy trained a machine to “watch” Blade Runner” · View on HN

MRI technology is improving over time, but I don't know how practical it would be in everyday usage. fNIR can potentially do the same thing much cheaper. I believe it's restricted to only the first few cm of brain tissue but that might be plenty. With fMRI you can even reconstruct entire videos of what a person is seeing (e.g. ▶ this video ). Controlling something simpler should be easier.

in “Introduction to Modern Brain-Computer Interface Design” · View on HN

Although it's hard to tell from the images presented with the article, the face generation looks like it could be similar to the techniques used in Nishimoto et al., 2011, which used a similar library of learned brain responses, though for movie trailers: ▶ this video Their particular process is described in the YouTube caption: The left clip is a segment of a Hollywood movie trailer that the subject viewed while in the magnet. The right clip shows the reconstruction of this segment from brain activity measured using fMRI. The procedure is as follows: [1] Record brain activity while the subject watches several hours of movie trailers. [2] Build dictionaries (i.e., regression models) that translate between the shapes, edges and motion in the movies and measured brain activity. A separate dictionary is constructed for each of several thousand points at which brain activity was measured. (For experts: The real advance of this study was the construction of a movie-to-brain activity encoding model that accurately predicts brain activity evoked by arbitrary novel movies.) [3] Record brain activity to a new set of movie trailers that will be used to test the quality of the dictionaries and reconstructions. [4] Build a random library of ~18,000,000 seconds (5000 hours) of video downloaded at random from YouTube. (Note these videos have no overlap with the movies that subjects saw in the magnet). Put each of these clips through the dictionaries to generate predictions of brain activity. Se

in “Yale researchers reconstruct facial images locked in a viewer’s mind” · View on HN

This is one year and something old and returns better results, in my opinion. ▶ this video On the other hand, they presented only the best results in that video.

in “Reconstruction Of Visual Images Using Brain Activity Patterns” · View on HN

According to the abstract: Here, we present a neural decoding approach in which machine learning models predict the contents of visual imagery during the sleep onset period given measured brain activity, by discovering links between human fMRI patterns and verbal reports with the assistance of lexical and image databases. Decoding models trained on stimulus-induced brain activity in visual cortical areas showed accurate classification, detection, and identification of contents. So it sounds like the fMRI patterns are associated with words via verbal reports from humans, and the video is simply generated from image databases by using those words. In other words, they classify the fMRI patterns based on how well they correspond to various words (measured while the subjects are awake and giving verbal descriptions of images they're looking at), and then the program measures the subjects' fMRI patterns while they're dreaming, determines the strongest 'word signals' in the patterns, and selects images from a photo database corresponding to those words and assembles them into a video based on how strong the similarity score for each word is at each moment in time. It's interesting, but the video seems like more of a "reenactment" of a dream than a prediction. It doesn't look like you can infer actions and occurrences within a dream from this. But this is all just my interpretation. edit: This is a fundamentally different approach from the one used in other research (e.g. ▶ this video me

in “Reconstruction Of Visual Images Using Brain Activity Patterns” · View on HN

Select the 100 clips whose predicted activity is most similar to the observed brain activity. Average these clips together. This is the reconstruction. Mind to explain to me why the right clip is inconsistent with the left clip? ▶ this video Starting at 20-second I feel like the right clip is out of touch with the left clip. between 20th and 22nd second I see at least three individuals rendered from the reconstruction. From 26th to the end of the clip I also see multiple individuals. The names also look different from one another... When you say find the closest is that an expected result?

in “Yale researchers reconstruct facial images locked in a viewer’s mind” · View on HN