WEBVTT 00:06.440 --> 00:10.090 Um what's the job of CD A O ? 00:10.329 --> 00:13.239 This is really if I can , this is 00:13.250 --> 00:15.880 really a , a tough call . Uh We were 00:15.890 --> 00:18.280 joined and all of , you know , and by 00:18.290 --> 00:20.620 the ruins in the audience . So uh who , 00:20.629 --> 00:22.962 who ran the Jake ? Uh we were , we were , 00:22.962 --> 00:24.962 we were the merger of four distinct 00:24.962 --> 00:28.540 doors , the Jake , the CEO S office 00:30.139 --> 00:31.917 A data , which is the analytics 00:31.917 --> 00:33.750 platform in DD defense , digital 00:33.750 --> 00:36.029 Services . And the idea was to put all 00:36.040 --> 00:38.096 of those together , essentially have 00:38.360 --> 00:41.229 cies of scale . So the job of that CD A 00:41.240 --> 00:44.590 O office , I think we call it the O CD 00:44.599 --> 00:47.110 O si I , I'm still not very good with 00:47.119 --> 00:50.240 dndis , but the job of the office is to 00:50.250 --> 00:53.389 scale data analytics and A I 00:54.000 --> 00:56.659 across the department for decision 00:56.669 --> 01:00.000 advantage . Uh I think I should say 01:00.009 --> 01:02.065 from the boardroom to the dialogue , 01:02.065 --> 01:04.231 you know , for decision advantage . So 01:04.629 --> 01:06.851 the key part there that I added , which 01:06.851 --> 01:09.073 wasn't part of the original mission was 01:09.073 --> 01:12.919 scale and the scaling part is the hard 01:12.930 --> 01:16.160 part . So what I discovered when I 01:16.169 --> 01:18.336 first came up uh into the department , 01:18.336 --> 01:20.447 almost a couple of things . The first 01:20.447 --> 01:22.558 thing I discovered , I know my team's 01:22.558 --> 01:22.099 all frustrated me already because I'm 01:22.110 --> 01:25.449 not taking the script at all . Um The 01:25.459 --> 01:27.626 first thing I , and they worked really 01:27.626 --> 01:29.348 hard on . Um The first thing I 01:29.348 --> 01:32.709 discovered um was what I thought was 01:32.720 --> 01:35.389 the problem , wasn't the problem . What 01:35.400 --> 01:37.511 I thought the problem was going to be 01:37.511 --> 01:39.870 is creating a demand signal for being 01:39.879 --> 01:42.169 digitally driven , creating a demand 01:42.260 --> 01:44.989 signal for being data driven . Not true . 01:45.089 --> 01:47.089 I'm unbelievably impressed . I , so 01:47.089 --> 01:49.200 I've been in the job for about a year 01:49.200 --> 01:51.422 and a half . The first six months don't 01:51.422 --> 01:53.645 count because I don't know if anybody's 01:53.645 --> 01:52.949 worked in the building , but it takes 01:52.959 --> 01:56.889 that long on that . Um So over the last 01:56.900 --> 02:00.269 year when I discovered uh is that 02:00.279 --> 02:02.279 everybody wants to be data driven , 02:02.279 --> 02:04.610 everybody wants the right data at the 02:04.620 --> 02:06.676 right time to solve the problem . In 02:06.676 --> 02:09.000 fact , everybody wants it so badly . 02:09.130 --> 02:11.770 They are willing to believe in that . 02:12.050 --> 02:14.272 We'll get down in a second . I want you 02:14.272 --> 02:16.449 to kind of , and everybody wants to be 02:16.460 --> 02:19.169 in the B so that wasn't my problem 02:19.500 --> 02:21.722 trying to convince people to be dated . 02:21.722 --> 02:23.919 It was my problem trying to help them 02:23.929 --> 02:26.350 figure out , figure out how to be d 02:27.419 --> 02:30.800 that's the problem . So , and , and , 02:30.809 --> 02:33.031 and there's two ways in which we have a 02:33.149 --> 02:36.990 problem . Why was this tension ? 02:37.339 --> 02:40.759 Don't worry . I , why was this tension 02:41.550 --> 02:44.759 between the fight tonight and getting 02:44.770 --> 02:48.020 it ? Right . No . What I mean by 02:48.029 --> 02:50.085 getting correct is getting a correct 02:50.085 --> 02:53.529 department . So , and I just chose that , 02:54.399 --> 02:58.139 um , the department is 02:58.149 --> 03:00.960 filled with patriots . These are people 03:00.970 --> 03:04.699 who really want to save lives , who 03:04.710 --> 03:07.320 really want to do the right thing right 03:07.330 --> 03:10.190 away . Well , sometimes doing the right 03:10.199 --> 03:12.460 thing right away means the solution is 03:12.470 --> 03:15.149 happening . Actually , I've been doing 03:15.160 --> 03:17.271 the right . Right . Right . Means the 03:17.271 --> 03:19.669 solutions happen . And I will admit the 03:19.679 --> 03:21.990 department has a tendency to quickly 03:22.000 --> 03:24.320 hack together a solution and move on to 03:24.330 --> 03:26.699 the next emergency . Why is it the US 03:26.710 --> 03:28.599 Department of Defense , there are 03:28.599 --> 03:30.710 always emergencies , there are always 03:30.710 --> 03:32.932 things going on that need to be fixed . 03:32.932 --> 03:36.410 So we took it as our task , tobacco . 03:36.419 --> 03:39.160 What does it mean to get it right . 03:39.169 --> 03:41.336 Three years from now to now about what 03:41.336 --> 03:43.336 does it mean to get it right . Five 03:43.336 --> 03:45.280 years from now . Does that mean we 03:45.280 --> 03:47.447 don't respond ? Should we fix that for 03:47.447 --> 03:49.780 five years now ? We have to respond now . 03:49.780 --> 03:49.460 But if we have a view of what it means 03:49.470 --> 03:51.581 to be , right ? Five years from now , 03:51.581 --> 03:53.637 or three years from now or two years 03:53.637 --> 03:55.803 from now , then once we do a quick and 03:55.803 --> 03:58.026 happy solution , we know that it's tech 03:58.026 --> 04:01.669 dead and we know how to modify it to 04:01.679 --> 04:03.901 make it be a sustainable solution . The 04:03.901 --> 04:06.123 way I say this and I'm sure some of you 04:06.123 --> 04:08.235 have heard said this before , but the 04:08.235 --> 04:10.346 way I say this to my team is , look , 04:10.346 --> 04:12.512 our job is not only to answer the call 04:12.512 --> 04:14.623 today . And that is our job to answer 04:14.623 --> 04:16.790 the call today . And as you go through 04:16.790 --> 04:18.957 this , uh , symposium , you'll see our 04:18.957 --> 04:21.179 search capabilities . You'll see plenty 04:21.179 --> 04:23.179 of evidence of our jumping into the 04:23.179 --> 04:25.346 fight right now . It's actually pretty 04:25.346 --> 04:27.989 exciting . It's actually our job to 04:28.000 --> 04:30.222 answer the phone . It answered the call 04:30.222 --> 04:32.333 today . But how many of you have ever 04:32.333 --> 04:34.556 used the government system ? Raise your 04:34.556 --> 04:37.209 hand . Ok . How many of you have ever 04:37.220 --> 04:40.070 used the governance system ? It wasn't 04:40.079 --> 04:42.540 quite that far . Yeah , you all are 04:42.549 --> 04:43.771 using PTS to get your 04:46.779 --> 04:48.446 luckily this iteration of the 04:48.446 --> 04:50.612 government . I don't have to use UTS . 04:50.612 --> 04:54.589 But uh so , so what if you were using a 04:54.600 --> 04:57.260 system in a forward base ? What if you 04:57.269 --> 05:00.410 were using a system in a combat ? I see 05:00.420 --> 05:02.309 a lot of military folks here . So 05:02.309 --> 05:04.309 they're gonna understand this . And 05:04.309 --> 05:06.476 what if you could make a phone call to 05:06.476 --> 05:08.698 five years ago ? And you could ask that 05:08.698 --> 05:10.753 one , the developer of that system . 05:10.753 --> 05:12.920 Hey , I know you're in a rush . I know 05:12.920 --> 05:15.142 you're solving an immediate problem . I 05:15.142 --> 05:17.309 know you're solving a problem . That's 05:17.309 --> 05:19.364 gonna say life right now . But could 05:19.364 --> 05:21.476 you just make this one tweet ? If you 05:21.476 --> 05:23.476 could make this one tweet ? My life 05:23.476 --> 05:25.476 would be so much better . In fact , 05:25.476 --> 05:25.299 everything would be much more efficient . 05:25.970 --> 05:28.081 So we take it as our responsibility , 05:28.081 --> 05:30.192 not just to take the call all night , 05:30.192 --> 05:32.359 which we will , but to take that call 05:32.369 --> 05:34.480 from five or three or two years ago . 05:34.480 --> 05:36.591 Two years from now , we're gonna take 05:36.591 --> 05:38.758 that future phone call as well because 05:38.758 --> 05:42.690 we are gonna be successful . I'm 05:42.700 --> 05:44.811 not sure if my team buys this as much 05:44.811 --> 05:46.920 as I can , but we're gonna be 05:46.929 --> 05:50.100 successful three years from now . If 05:50.109 --> 05:52.276 we're all standing in front of a bunch 05:52.276 --> 05:54.880 of bricks on the ground with bloody 05:54.890 --> 05:57.010 foreheads from beating our heads 05:57.019 --> 05:59.920 against those bricks . That's really my 05:59.929 --> 06:02.790 problem by having this brick wall , but 06:02.799 --> 06:05.021 we broke through that wall , the bricks 06:05.021 --> 06:07.109 are everywhere but everybody else 06:07.119 --> 06:10.829 thinks it was their idea that's winning 06:10.839 --> 06:13.790 for us , winning for us as everybody 06:13.799 --> 06:16.559 else thinks . Oh , I launched a I , I 06:16.570 --> 06:19.690 solved this data problem . I quickly 06:19.700 --> 06:21.756 leverage data to build an analytical 06:21.760 --> 06:24.119 solution that solved my commanders 06:24.130 --> 06:27.299 problem right away and I have the tools 06:27.440 --> 06:29.809 I have the infrastructure . I have the 06:29.820 --> 06:32.269 policies and I have the contract 06:32.279 --> 06:36.010 vehicles to deliver it . That's many . 06:36.739 --> 06:38.850 It's not quite where we are right now 06:38.850 --> 06:40.683 where we are right now is we can 06:40.683 --> 06:42.795 quickly have things together , we can 06:42.795 --> 06:44.850 quickly contract something , but the 06:44.850 --> 06:47.510 specs might not be as well thought out 06:47.519 --> 06:49.519 for the law as they need to be even 06:49.519 --> 06:51.686 though they're really well thought out 06:51.686 --> 06:54.790 for the immediate fight . And so , so 06:54.799 --> 06:57.839 our goal , a really important goal and 06:58.079 --> 07:00.429 look , I have this battle within , 07:00.440 --> 07:02.709 within the org and I have this battle 07:02.869 --> 07:05.230 within the department . There's a large 07:05.239 --> 07:09.070 fashion that wants us to focus only on 07:09.079 --> 07:10.890 the fight tonight and I totally 07:10.899 --> 07:13.619 understand that I've never been a war 07:13.630 --> 07:15.741 fighter but I was pressured . FP si , 07:15.741 --> 07:18.130 have a , for 12 years . I have a sense 07:18.140 --> 07:21.190 of the way war fighters think marines . 07:21.200 --> 07:23.200 I'm sure you're in the audience . I 07:23.200 --> 07:25.256 definitely have a sensible way . You 07:25.256 --> 07:24.239 think ? It's fascinating . 07:29.500 --> 07:31.667 Hey , are you here ? You can get at me 07:31.667 --> 07:31.450 later . 07:35.709 --> 07:37.876 So I've never gonna work by it , but I 07:37.876 --> 07:40.042 really get that right . I get it . But 07:40.042 --> 07:42.049 I also get using the marine analogy 07:42.170 --> 07:45.290 that if the backpack , the backpack is 07:45.299 --> 07:48.290 uh only so big and if it contains that 07:48.299 --> 07:50.521 doesn't work , it's gonna get , that is 07:50.521 --> 07:53.420 gonna get left in the sand . So we're 07:53.429 --> 07:55.262 trying really hard to strike the 07:55.262 --> 07:57.640 balance between the bi night and the 07:57.649 --> 08:00.040 long term giving advice . That that's 08:00.049 --> 08:03.790 my , how did we approach this ? You all 08:03.799 --> 08:05.577 heard this as well ? How did we 08:05.577 --> 08:08.170 approach this ? I asked when we first 08:08.179 --> 08:10.130 arrived ? Well , what's the most 08:10.140 --> 08:12.251 important thing that we have to build 08:12.450 --> 08:14.690 if we're going to build ? Now , one of 08:14.700 --> 08:16.811 the reasons for merging data analysis 08:16.811 --> 08:20.600 and A I guess uh it , it is so that we 08:20.609 --> 08:23.570 can have a unified strategy across data 08:23.579 --> 08:25.880 and , and A I , well , the problem with 08:25.890 --> 08:27.946 Unified Strategy , Michael Brown , I 08:27.946 --> 08:29.946 think you're gonna agree with me on 08:29.946 --> 08:32.001 this general . I'm not sure we are , 08:32.001 --> 08:34.168 but I saw you earlier . Uh There he is 08:34.280 --> 08:36.391 the problem with the Unified Strategy 08:36.391 --> 08:38.690 is one part wins . And another part 08:38.700 --> 08:40.820 suffers if you have three distinct 08:40.830 --> 08:42.886 organizations , they all get to push 08:42.886 --> 08:44.830 their strategy . But for a unified 08:44.830 --> 08:47.163 strategy against data analytics and A I , 08:47.163 --> 08:48.941 well , then we have to actually 08:48.941 --> 08:51.299 prioritize . So I I wish my view is the 08:51.309 --> 08:53.476 right thing to do . We should actually 08:53.476 --> 08:55.531 build the right thing first . Well , 08:55.531 --> 08:57.365 ideally not slowing anybody down 08:57.365 --> 08:59.531 because that's a tough balance . I try 08:59.531 --> 09:01.476 really hard on that one . That's a 09:01.476 --> 09:03.698 tough balance . So what is building the 09:03.698 --> 09:05.865 right first ? Well , we thought really 09:05.865 --> 09:07.920 hard about this hierarchy of means . 09:07.920 --> 09:07.739 Not sure you all have heard me talk 09:07.750 --> 09:11.479 about and the h he has in the bottom 09:12.460 --> 09:16.130 data quality . Why do we have data 09:16.140 --> 09:18.900 quality as the , the , the the 09:18.909 --> 09:20.853 foundational thing that we have to 09:20.853 --> 09:23.739 build is not A I . Well , remember my 09:23.750 --> 09:27.530 goal is this balance between the long 09:27.539 --> 09:29.539 term getting and right in the fight 09:29.539 --> 09:31.706 tonight . So if we only cared about by 09:31.706 --> 09:33.872 tonight , that data quality isn't that 09:33.872 --> 09:35.928 important ? Long term data quality . 09:35.928 --> 09:37.706 Isn't that important ? We could 09:37.706 --> 09:40.059 actually just contract out a stovepipe 09:40.070 --> 09:42.181 solution . And for my entire career , 09:42.181 --> 09:44.014 I've heard people talk about how 09:44.014 --> 09:45.959 horrible stovepipe solutions are . 09:45.959 --> 09:48.126 However , if we're only gonna focus on 09:48.126 --> 09:50.348 the fight tonight , what do you think ? 09:50.348 --> 09:53.169 We buy stovepipe solutions ? I get it . 09:53.609 --> 09:55.720 I I this is not a criticism . This is 09:55.720 --> 09:57.998 an analysis of how we got where we are . 09:57.998 --> 10:00.419 We want to go fast . Someone has an 10:00.429 --> 10:02.390 answer . We buy that answer . That 10:02.400 --> 10:04.511 answer is a particular answer . Not a 10:04.511 --> 10:07.250 Generali . And so we buy another ticket , 10:07.260 --> 10:09.489 answer , not a general liable one . And 10:09.500 --> 10:11.722 the navy buys a third and the army buys 10:11.722 --> 10:13.667 a four and they don't talk to each 10:13.667 --> 10:15.667 other . Why ? Because we were going 10:15.667 --> 10:19.260 fast , going fast is good . Building in 10:19.270 --> 10:22.049 a scalable way while still going faster . 10:22.880 --> 10:25.102 So that's what we're trying to do . And 10:25.102 --> 10:27.520 so data quality is the most important 10:27.530 --> 10:31.510 aspect of that because what is A I 10:31.520 --> 10:34.020 without quality data ? What's a I ? 10:35.900 --> 10:38.869 Mhm Anybody 10:44.109 --> 10:46.165 did you ask have a person to leave , 10:46.165 --> 10:49.929 please be be what is 10:49.940 --> 10:52.539 artificial intelligence ? A big pile of 10:52.549 --> 10:55.640 math makes guesses a big pile of math 10:55.650 --> 10:59.159 and makes guesses wrong . It's a big 10:59.169 --> 11:01.729 pile of math . The next guess is using 11:01.739 --> 11:04.510 a big pile of paper . That's the 11:04.520 --> 11:06.576 process I shouldn't said right ? You 11:06.576 --> 11:08.631 are 100% right ? I really appreciate 11:08.631 --> 11:09.631 you being here . 11:12.099 --> 11:15.530 So for us A I statistics at 11:15.539 --> 11:17.809 scale and what does it mean to be 11:17.820 --> 11:21.049 statistics at scale ? It means we count 11:21.059 --> 11:23.226 the past . I'm sure you've always said 11:23.226 --> 11:25.392 this before , but we count the past to 11:25.392 --> 11:27.750 predict the future . How do you counter 11:27.760 --> 11:31.489 that ? You have high quality data 11:31.739 --> 11:33.683 that tells you what the past is up 11:35.460 --> 11:37.710 on top of the high quality data layer . 11:37.719 --> 11:39.608 We have the analytics and metrics 11:39.950 --> 11:43.169 lately and this is why that is such an 11:43.179 --> 11:45.179 important part of being part of the 11:45.179 --> 11:47.290 team because I I uh not only drives a 11:47.290 --> 11:49.346 huge part of the high quality data , 11:49.346 --> 11:51.401 but also drives the analytic layer , 11:51.401 --> 11:53.568 those those dashboards that we need to 11:53.568 --> 11:55.790 figure out uh how many things we have , 11:55.790 --> 11:57.846 what we have , where we have , where 11:57.846 --> 11:59.790 are our troops , you know , uh I'm 11:59.950 --> 12:02.880 being repetitive , but I'm pretty sure 12:02.890 --> 12:06.830 80% of the demand for A I is really 12:06.840 --> 12:08.840 satisfiability by high quality data 12:08.960 --> 12:11.071 dashboard . So you can see where your 12:11.071 --> 12:13.909 stuff is . So that's the next layer and 12:13.919 --> 12:17.429 then find A I on top of that . Now , I 12:17.440 --> 12:20.719 don't mean this hierarchy of needs to 12:20.729 --> 12:24.099 be uh a terrible hierarchy . I don't 12:24.109 --> 12:26.109 mean it to be oh like my piece , we 12:26.109 --> 12:28.165 have a whole bunch of enablers under 12:28.165 --> 12:32.109 the high quality . I see you . We 12:32.119 --> 12:34.230 have a whole bunch of neighbors under 12:34.230 --> 12:36.175 the high quality . Those neighbors 12:36.175 --> 12:39.479 include this fantastic CIU Cio . 12:39.690 --> 12:42.320 Uh um what's the government cloud 12:43.049 --> 12:46.380 JWCC ? So I work out our contract . If 12:46.390 --> 12:48.446 we didn't have that , we wouldn't be 12:48.446 --> 12:50.501 able to build that stuff . I mean we 12:50.501 --> 12:52.668 could , but this is really gonna allow 12:52.668 --> 12:54.557 us to scale . So having the right 12:54.557 --> 12:56.723 contracts for the right infrastructure 12:56.723 --> 12:58.723 that's extremely important . Uh How 12:58.723 --> 13:01.929 about talent as we are the functional 13:01.940 --> 13:05.630 community manager for 11 13:05.640 --> 13:09.500 new te 11 new uh tech roles , 13:09.510 --> 13:11.677 digital roles within the department of 13:11.677 --> 13:14.239 that , which include data analysts , uh 13:14.309 --> 13:17.840 um data scientist , data engineer , uh 13:17.849 --> 13:20.169 A I AM L scientist . There's a whole 13:20.179 --> 13:24.140 bunch 10 distinct um new roles 13:24.150 --> 13:27.799 and you'll be shy of the power of 13:27.809 --> 13:30.200 simply going and , and uh well , you'll 13:30.210 --> 13:32.099 see this as you go throughout the 13:32.099 --> 13:34.099 symposium and the power of actually 13:34.099 --> 13:36.210 just telling people that what they're 13:36.210 --> 13:38.099 actually doing is this new role . 13:38.130 --> 13:39.908 Because if you code these roles 13:39.908 --> 13:42.130 correctly , if you code bills correctly 13:42.130 --> 13:44.019 for these new roles . Well , what 13:44.019 --> 13:46.130 happens ? Well , organically suddenly 13:46.130 --> 13:48.241 can under those rules , organically , 13:48.241 --> 13:50.309 people hire people uh against the 13:50.320 --> 13:52.640 requirements for people who can , can 13:52.650 --> 13:55.919 do those things . And simultaneously we 13:55.929 --> 13:58.140 provide the training to executives , 13:58.150 --> 14:00.261 the training to middle management and 14:00.261 --> 14:02.206 Pythons from schools , then we can 14:02.206 --> 14:04.372 start to see talent across the whole . 14:04.690 --> 14:06.729 And that's a big part of CB a job . 14:08.979 --> 14:12.559 How about you , sir ? Like 14:12.849 --> 14:13.869 1015 minutes . 14:16.919 --> 14:18.419 So why are we doing this ? 14:22.419 --> 14:25.830 Imagine the world where combatant 14:25.919 --> 14:28.239 commands can see everything they need 14:28.250 --> 14:30.361 to see the next strategic decisions . 14:31.250 --> 14:34.429 Imagine a world where those Kabbalah 14:34.469 --> 14:37.679 commanders aren't getting that 14:37.690 --> 14:41.390 information via powerpoint are getting 14:41.400 --> 14:45.330 that information via email across 14:45.559 --> 14:48.239 uh or the turnaround time for 14:48.250 --> 14:50.361 situational awareness of what's going 14:50.361 --> 14:53.270 on . Shrinks from a day or 2 to 10 14:53.280 --> 14:56.280 minutes . How do we build that ? 14:57.210 --> 14:59.377 Well , we will that work lots of way . 14:59.377 --> 15:02.020 So one of the key things is uh high 15:02.030 --> 15:04.659 quality data that's accessible , 15:04.900 --> 15:08.869 discoverable , accessible and available 15:08.880 --> 15:11.030 to you to build an app on top of it 15:11.039 --> 15:13.261 very quickly in a way that gotten about 15:13.549 --> 15:17.219 music . How in the world where the 15:17.229 --> 15:20.030 view that the bath man has doesn't stop 15:20.039 --> 15:23.380 the boundaries of their A or because I 15:23.390 --> 15:25.557 don't know if you look at that but the 15:25.557 --> 15:27.779 boundaries of one A or is the beginning 15:27.779 --> 15:29.723 of the next A or , and things move 15:29.723 --> 15:29.619 across those boundaries all the time , 15:30.080 --> 15:32.280 wouldn't it be great ? The same data 15:32.289 --> 15:35.080 layer , the same data layer that 15:35.090 --> 15:38.039 provided setcom with a view of the 15:38.049 --> 15:41.010 world also provided you with the same 15:41.020 --> 15:43.131 view of the world . Now , you doesn't 15:43.131 --> 15:45.353 have to look across everything but they 15:45.353 --> 15:47.576 can , they can track things that may be 15:47.576 --> 15:49.631 coming in their way . They can track 15:49.631 --> 15:51.909 things that set c say , hey , you work , 15:52.039 --> 15:55.640 right ? And uh uh from San , you could 15:55.650 --> 15:57.817 go through a canal or two and suddenly 15:57.817 --> 16:00.799 you're in into pay off . Well , that's 16:00.809 --> 16:02.920 great knowing that that's happening . 16:02.920 --> 16:05.330 Knowing that allowing for that data to 16:05.340 --> 16:07.619 easily flow and be picked up by the 16:07.820 --> 16:10.909 combat man , that's what we mean by 16:10.919 --> 16:12.919 high quality data . And we see this 16:12.919 --> 16:16.919 high quality data in a database . This 16:16.929 --> 16:19.151 is where it's really valuable for , for 16:19.151 --> 16:21.318 an industry to help us out . We really 16:21.318 --> 16:23.540 need your help . What do we mean by the 16:23.540 --> 16:25.540 mesh ? I've said this before . It's 16:25.540 --> 16:27.596 still a buzz phrase . I'm gonna tell 16:27.596 --> 16:29.849 exactly what I mean by it that the data 16:29.859 --> 16:32.909 stays where the data is . It's not 16:32.919 --> 16:35.020 necessarily , you can put it into a 16:35.030 --> 16:37.141 light . I don't care , but I mostly I 16:37.141 --> 16:39.390 don't care where the data is and the 16:39.400 --> 16:41.909 best place for the data is with the 16:41.919 --> 16:44.150 owners of the data . People understand 16:44.159 --> 16:46.492 the data , the most understand the best . 16:46.940 --> 16:49.380 That's number one , number two is 16:49.390 --> 16:52.179 successful via an API and we've been 16:52.190 --> 16:54.409 working really hard with both industry 16:54.419 --> 16:56.363 and our body to make it as easy as 16:56.363 --> 16:58.475 possible to find data through the big 16:58.475 --> 17:00.570 diagram . Number three , it's 17:00.580 --> 17:02.900 essentially discovered . I think that's 17:02.909 --> 17:04.909 the one major thing that government 17:04.909 --> 17:07.020 owns in this story . Government owns 17:07.030 --> 17:09.619 the the catalog , government owns the 17:09.630 --> 17:12.920 catalog and the list of metadata that 17:12.930 --> 17:14.930 as you publish your data , I have a 17:14.930 --> 17:17.041 data set . This data set is available 17:17.041 --> 17:20.800 the following API here's the meta data 17:20.810 --> 17:23.479 about my data set . And number four , 17:23.489 --> 17:26.030 it's owned by somebody as a product . 17:26.310 --> 17:28.729 Here's the person you talk to if you 17:28.739 --> 17:31.780 want access to this data . Now the 17:31.790 --> 17:33.957 policy teams can tell me there's a lot 17:33.957 --> 17:35.734 of order a data match , there's 17:35.734 --> 17:37.679 computational governance , there's 17:37.679 --> 17:39.901 computational analysis of make sure the 17:39.901 --> 17:41.901 policies are in place . And I think 17:41.901 --> 17:41.890 that's great , but these are the things 17:41.900 --> 17:44.910 that drive it from . Why do we want 17:44.920 --> 17:46.976 ownership ? Let me , let me , let me 17:46.976 --> 17:49.310 pull that thread for a second again , 17:49.319 --> 17:51.375 being repetitive . I think it really 17:51.375 --> 17:53.375 worth it's really worth repeating . 17:55.760 --> 17:57.760 That's a great question . I have an 17:57.760 --> 17:59.000 answer to it . So 18:02.229 --> 18:04.285 we traditionally say that data is an 18:04.285 --> 18:08.199 asset and I think that's true but 18:08.209 --> 18:10.265 it's not , doesn't come to the whole 18:10.265 --> 18:12.689 story . Data has an asset . Well , an 18:12.699 --> 18:16.119 asset is fungible data is not data 18:16.130 --> 18:19.969 specific an asset uh is it has 18:19.979 --> 18:22.201 value in it of itself . So the more you 18:22.201 --> 18:24.479 have , the more value you have . That's 18:24.489 --> 18:26.660 not true of that at all . I don't care 18:26.670 --> 18:28.892 how much that you have . No one's using 18:28.892 --> 18:31.510 it . It's useless . Data is only 18:31.520 --> 18:34.459 valuable if it has customers . 18:35.500 --> 18:37.589 But are customers , customers are 18:37.599 --> 18:40.459 people to whom you make process , the 18:40.469 --> 18:42.691 data will be of the following quality . 18:42.691 --> 18:44.580 The data will be of the following 18:44.580 --> 18:46.969 latency . There will be back bills 18:46.979 --> 18:50.189 within a month . Whatever the promises 18:50.199 --> 18:52.430 you make about your data set as the 18:52.439 --> 18:54.495 data owner . So we call it data as a 18:54.495 --> 18:56.800 product and the owner is a product 18:56.810 --> 19:00.189 manager that whatever promises you make , 19:00.199 --> 19:02.366 your customers should be able to count 19:02.366 --> 19:04.088 on them and then you should be 19:04.088 --> 19:06.310 evaluated by delivering against that . 19:07.000 --> 19:09.167 What we , what we evaluate now is , do 19:09.167 --> 19:12.630 you have a bunch of data ? Ok . Not so 19:12.640 --> 19:15.130 great . I think what we should evaluate 19:15.140 --> 19:17.339 against is how many customers are 19:17.349 --> 19:19.349 getting value from your data ? What 19:19.349 --> 19:21.489 value are they getting ? What , what 19:21.500 --> 19:23.556 feedback can I get to your customers 19:23.556 --> 19:25.722 that tell me that without their data , 19:25.722 --> 19:27.944 they couldn't have won that data , they 19:27.944 --> 19:29.889 couldn't have done their job . The 19:29.889 --> 19:31.889 value of the data is the value that 19:31.889 --> 19:34.400 brings to the customer serves and ask 19:34.410 --> 19:36.632 the question . That's the definition of 19:36.632 --> 19:38.466 quality . We're not gonna have a 19:38.466 --> 19:40.410 unified . So the question is , how 19:40.410 --> 19:42.466 would you modify high quality data , 19:42.466 --> 19:44.688 garbage , garbage out . And what if you 19:44.688 --> 19:46.743 don't have quality data ? Well , the 19:46.743 --> 19:46.589 way we have to do this is to sign 19:46.599 --> 19:48.710 ownership each of these data products 19:48.750 --> 19:51.369 and that person who owns it , their job 19:51.380 --> 19:54.109 rating is dependent upon it , serving 19:54.119 --> 19:56.290 value for the customers of that data . 19:56.560 --> 19:58.282 And if it serves value for the 19:58.282 --> 20:00.393 customers with that data , that's how 20:00.393 --> 20:03.050 we OK . 20:05.560 --> 20:08.130 So high quality data and data mesh 20:09.810 --> 20:13.719 uh combat demands one example , another 20:13.729 --> 20:16.079 example . But if we have data , not 20:16.089 --> 20:18.770 just uh from the battlefield , but from 20:18.780 --> 20:20.891 the board room to the battlefield and 20:20.891 --> 20:23.540 it's all in this data mesh . And uh 20:23.550 --> 20:25.717 well , the talent board has done their 20:25.717 --> 20:28.199 job like I just described . So there's , 20:28.209 --> 20:30.376 there's sufficiently trained , I won't 20:30.376 --> 20:32.510 say highly trained , sufficiently 20:32.520 --> 20:35.660 trained data and A I practitioners 20:36.040 --> 20:38.640 distributed across the course . And 20:38.650 --> 20:40.761 what if the policy has done their job 20:41.609 --> 20:45.180 and contracting has done their job and 20:45.189 --> 20:47.630 the infrastructure team has done their 20:47.640 --> 20:50.589 job . So that one of these well trained 20:50.689 --> 20:54.430 soldiers , sailors , guardians , airmen 20:54.439 --> 20:57.410 or marines uh 20:59.239 --> 21:01.329 sees a problem that wants to fix it 21:01.949 --> 21:03.949 right now . If they see a problem , 21:03.949 --> 21:06.469 they wanna fix it . It's a , it's 21:06.479 --> 21:09.229 either slightly un Kosher hacked 21:09.239 --> 21:12.819 together or it can't be done . We wanna 21:12.829 --> 21:14.979 make it as easy as possible for those 21:14.989 --> 21:17.890 closest to the problem to find the data , 21:17.920 --> 21:20.750 have the tools have the policies have , 21:21.150 --> 21:23.609 have the authorization authority to be 21:23.619 --> 21:25.750 able to quickly build a solution . I 21:25.760 --> 21:27.538 didn't say that quickly build a 21:27.538 --> 21:30.390 solution , quickly run it from TNE , 21:30.400 --> 21:33.339 quickly get approval to operate and 21:33.349 --> 21:35.627 quickly deploy it in a sustainable way . 21:36.359 --> 21:38.581 That's really hard for those of you who 21:38.581 --> 21:40.637 don't know all of the pieces there . 21:40.637 --> 21:42.803 That's extremely hard . But that's our 21:42.803 --> 21:44.692 vision , our vision is to get the 21:44.692 --> 21:46.803 policy right , to get the authorizing 21:46.803 --> 21:49.026 the authorities to operate , right , to 21:49.026 --> 21:51.026 get acquisitions right , to get the 21:51.026 --> 21:53.137 infrastructure , right . So , so what 21:53.137 --> 21:55.359 do we invite infrastructure here here ? 21:55.359 --> 21:57.248 We , we , we talk a lot about A I 21:57.248 --> 21:59.470 scathing A I scaffolding . So why don't 21:59.470 --> 22:01.303 we , why don't we talk about A I 22:01.303 --> 22:00.420 scolding ? 22:07.109 --> 22:09.165 Uh I'll just stop . Can you guys not 22:09.165 --> 22:11.331 show me the questions because you show 22:11.331 --> 22:10.930 me the questions . I'm gonna be reading 22:11.109 --> 22:14.469 the whole time . Show it . Yeah . 22:15.020 --> 22:18.770 Uh I don't think it 22:18.780 --> 22:22.209 makes that much sense for most 22:22.219 --> 22:26.180 problems for . Well , it's unclear to 22:26.189 --> 22:28.390 me . So I'll just say I'm not sure the 22:28.400 --> 22:30.511 degree to which the problem with that 22:30.680 --> 22:34.619 should be building A I why ? 22:34.989 --> 22:38.189 Because industry is great at that . We 22:38.199 --> 22:40.143 should be paying industry to build 22:40.143 --> 22:42.143 models because they are going to be 22:42.143 --> 22:44.199 always on the cutting edge . They're 22:44.199 --> 22:44.199 always gonna be doing the , they have 22:44.209 --> 22:46.489 the newest sexiest thing , language 22:46.500 --> 22:48.667 model , uh large language models , not 22:48.667 --> 22:51.140 standing the newest sexiest thing . 22:51.150 --> 22:53.206 They're always gonna have the newest 22:53.206 --> 22:55.317 possible solutions and they're always 22:55.317 --> 22:56.872 gonna be experimenting it , 22:56.872 --> 22:59.039 experimenting with these tools in ways 22:59.039 --> 23:01.261 that we are not gonna have time to . So 23:01.261 --> 23:01.069 let's just assume for a second that 23:01.079 --> 23:03.135 industry builds the models . I don't 23:03.135 --> 23:05.357 think that's always gonna be the case . 23:05.357 --> 23:07.468 I think we'll have citizen or soldier 23:07.468 --> 23:09.579 um uh data scientist who are building 23:09.579 --> 23:11.857 some models but most of the big models , 23:11.857 --> 23:13.801 um II , I don't think we should be 23:13.801 --> 23:15.579 building . I think it should be 23:15.579 --> 23:17.690 building . But what should we build ? 23:17.690 --> 23:19.968 We should own . Meaning the government , 23:20.339 --> 23:22.561 everything on the left and the right of 23:22.561 --> 23:25.119 the P . That's what I call ACOP . So 23:25.130 --> 23:27.408 what's on the left of the mouth ? Oops , 23:27.408 --> 23:29.463 sorry , left of the ball . What's on 23:29.463 --> 23:31.519 the left of the model are two of the 23:31.519 --> 23:33.979 hardest things that nobody wants to do . 23:34.829 --> 23:37.050 Nobody . What's the hardest part about 23:37.060 --> 23:38.227 building A I model ? 23:40.449 --> 23:43.390 Getting the data number one , getting 23:43.400 --> 23:45.829 the data . But people forget the 23:45.839 --> 23:48.250 following , getting the data . Doesn't 23:48.260 --> 23:50.316 mean just putting it all in the same 23:50.316 --> 23:52.371 directory because sometimes it's the 23:52.371 --> 23:54.316 PDF , sometimes it's the word do , 23:54.316 --> 23:56.739 sometimes it's Excel , sometimes it's 23:56.750 --> 23:58.910 actually in a structured database . 23:58.920 --> 24:01.250 Sometimes it's in a no SQL database . 24:01.270 --> 24:03.381 There's a million different ways that 24:03.381 --> 24:07.119 data can be . Should you and your team 24:07.130 --> 24:09.352 out there have to figure all that out . 24:09.459 --> 24:11.640 No , we want to find a contract with 24:11.650 --> 24:13.761 the tools , the capability for you to 24:13.761 --> 24:15.706 come to us . Uh not talking to the 24:15.706 --> 24:18.369 government folks uh to come to us and 24:18.380 --> 24:20.959 be able to quickly find the tooling 24:20.969 --> 24:23.670 they need or quickly contract with the 24:23.680 --> 24:25.791 vendors in the audience to be able to 24:25.791 --> 24:27.958 make that transformation for you . Get 24:27.958 --> 24:30.750 the deal . Labeling data is the part 24:30.760 --> 24:34.420 that everybody ignores but it is the 24:34.430 --> 24:37.760 IP that matters your labels . You as 24:37.770 --> 24:39.881 war fighters , I talk to war fighters 24:39.881 --> 24:42.369 for a second are the experts . You know 24:42.380 --> 24:45.079 what a means ? You know what B means ? 24:45.640 --> 24:47.696 The , the contractor sitting next to 24:47.696 --> 24:49.918 you might know something but they don't 24:49.918 --> 24:51.862 know it as well as you do . So the 24:51.862 --> 24:54.640 government , the military IP is the 24:54.650 --> 24:57.959 label data . It's also the hardest 24:57.969 --> 25:00.609 thing to do and nobody wants to do it . 25:01.000 --> 25:04.949 Nobody . Luckily we built contracts 25:04.959 --> 25:06.848 with companies like scale A I and 25:06.848 --> 25:10.520 others . Well , where uh members of the 25:10.530 --> 25:12.586 department can come to us , use that 25:12.586 --> 25:15.150 contract work through what data they 25:15.160 --> 25:17.216 need to solve their problem and they 25:17.216 --> 25:19.880 will help label that data for you . So 25:20.170 --> 25:23.030 industry , if you're gonna be helpful 25:23.040 --> 25:25.770 with data transformation , if you can 25:25.780 --> 25:28.500 be helpful with data labeling , if you 25:28.510 --> 25:30.566 could be helpful on the other side , 25:30.566 --> 25:32.899 data monitoring , say about that second , 25:32.899 --> 25:34.954 then that's how we need to work with 25:34.954 --> 25:37.040 you . We need to build the right 25:37.050 --> 25:39.280 contracts . Have a suite of companies 25:39.650 --> 25:41.761 that people inside the department can 25:41.761 --> 25:43.983 come to us and say here's the problem I 25:43.983 --> 25:46.119 want to solve and then we can solve 25:46.130 --> 25:48.297 with , here's the data that you need , 25:48.297 --> 25:50.408 here's how you have to transform that 25:50.408 --> 25:52.408 data and here's how you have to get 25:52.408 --> 25:54.630 like and if I just say to those folks , 25:54.630 --> 25:56.741 OK , see you later . Nothing will get 25:56.741 --> 25:58.908 done . So I'm gonna need you all to be 25:58.908 --> 26:01.074 there with me with the right contracts 26:01.074 --> 26:03.019 already in place to be able to say 26:03.019 --> 26:05.130 great money from that team will go on 26:05.130 --> 26:07.186 these contracts . But David labeling 26:07.186 --> 26:09.349 will get done , the data 26:09.359 --> 26:11.470 transformations will get done . Let's 26:11.470 --> 26:13.581 talk about model logic for a moment . 26:14.069 --> 26:18.040 One of the things that really struck me 26:18.050 --> 26:21.699 an industry even more so than the OD is 26:21.709 --> 26:24.800 how we shift and forget models , we 26:24.810 --> 26:27.140 just ship them and for them . Well , 26:27.150 --> 26:29.317 what's the problem with that ? What do 26:29.317 --> 26:31.483 they say in the beginning ? Like A I ? 26:31.483 --> 26:33.650 Is , is he taking notes for the hold ? 26:33.770 --> 26:35.770 Do you just write down notes for my 26:35.770 --> 26:39.150 whole ? Uh All right , a lot of 26:39.160 --> 26:40.993 language models , we were really 26:40.993 --> 26:43.520 involved with that . Actually , there's 26:43.530 --> 26:45.419 one where I think I see you a lot 26:45.419 --> 26:49.300 probably actually work great uh to 26:53.160 --> 26:55.989 thank you . Shouldn't forget that . 26:58.420 --> 27:02.030 So , so look , when we , when we build 27:02.040 --> 27:04.719 these models , we train them from the 27:04.729 --> 27:08.119 past , we ask about the past , what is 27:08.130 --> 27:10.297 A and what to speak ? Why ? Because we 27:10.297 --> 27:12.297 only have past data , we don't have 27:12.297 --> 27:14.408 data on the future , doesn't matter , 27:15.160 --> 27:18.609 sorry , doesn't exist . So we ask uh 27:18.619 --> 27:20.841 what's a , what's B , what's A , what's 27:20.841 --> 27:22.841 a speed we build the model ? We not 27:22.841 --> 27:24.897 don't forget A and B in that model . 27:25.250 --> 27:27.989 That story about , about training from 27:28.000 --> 27:30.359 the past to predict the future under 27:30.369 --> 27:32.313 what conditions will that model be 27:32.313 --> 27:35.689 successful . If the world in the 27:35.699 --> 27:38.219 future is sufficiently similar to the 27:38.229 --> 27:41.069 world in the past . If the world in the 27:41.079 --> 27:43.135 past is not sufficiently some of the 27:43.140 --> 27:45.029 world in the future , you model , 27:47.400 --> 27:51.290 if the world changes your model 27:51.300 --> 27:53.949 won't work or it won't work as well . 27:54.439 --> 27:57.449 We all forget this for some reason , we 27:57.459 --> 28:00.510 all forget this . We use model metrics 28:00.630 --> 28:04.469 like like accuracy or , or 28:04.479 --> 28:07.510 regression metrics , um like some of 28:07.520 --> 28:10.000 squared error root , mean squared error 28:10.270 --> 28:12.439 like that or we do things like f score 28:12.449 --> 28:14.920 precision recall . That's all great . 28:14.930 --> 28:16.874 But that's all . How well does the 28:16.874 --> 28:19.219 model fit the data in the past ? We 28:19.229 --> 28:21.709 have to continually monitor about 28:21.719 --> 28:23.830 whether that model is solving our use 28:23.830 --> 28:26.790 case in the future . Now , how many 28:26.800 --> 28:28.969 were writers in the room as you have 28:29.160 --> 28:33.020 this problem ? There you go uh in a 28:33.030 --> 28:35.890 battle . How much does the world stay 28:35.900 --> 28:39.119 the same ? Not very much 28:39.130 --> 28:42.630 stuff blows up , things change . 28:43.630 --> 28:46.160 Well , you if we were trying to drive a 28:46.170 --> 28:48.392 truck driving down the street while the 28:48.392 --> 28:50.448 notion of the street is something to 28:50.448 --> 28:52.559 change , it's got big craters in it , 28:52.559 --> 28:55.500 for example , right ? So , so assuming 28:55.670 --> 28:57.670 that the world is going to stay the 28:57.670 --> 28:59.859 same is falling . So we're working 28:59.869 --> 29:02.036 really hard . We don't have this yet . 29:02.036 --> 29:04.202 Look for this in 24 these other things 29:04.202 --> 29:06.147 uh uh data labeling as a service . 29:06.147 --> 29:08.619 We're close to release that broadly , 29:08.640 --> 29:10.900 but we already have that piloting and 29:10.910 --> 29:13.089 um we're just getting started on data 29:13.099 --> 29:16.260 transformation as a service . Um I know 29:16.270 --> 29:18.326 some folks got that emailed me about 29:18.326 --> 29:20.548 that . We , we're definitely want to do 29:20.548 --> 29:22.603 the transmission as a service . It's 29:22.603 --> 29:24.659 extremely important . Uh But in 2024 29:24.659 --> 29:26.714 please be on the lookout about model 29:26.714 --> 29:28.770 monitoring this service . Now , what 29:28.770 --> 29:28.630 does that mean for you ? What does that 29:28.640 --> 29:30.751 mean for your industry ? If you build 29:30.751 --> 29:32.918 this model , it's gonna have to have a 29:32.918 --> 29:34.807 container , it's gonna have to be 29:34.807 --> 29:37.029 containerized in particular ways . It's 29:37.029 --> 29:39.251 gonna have to have a particular books . 29:39.251 --> 29:38.839 It does have to use particular 29:38.849 --> 29:41.410 infrastructure . Don't care which 29:41.420 --> 29:43.198 infrastructure folks across the 29:43.198 --> 29:45.209 department use don't have a strong 29:45.219 --> 29:47.330 opinion that everybody has to use the 29:47.330 --> 29:49.386 same model , training pipeline , the 29:49.386 --> 29:51.608 same model for the pipeline do not have 29:51.608 --> 29:53.497 a strong opinion at all . I would 29:53.497 --> 29:56.359 rather let heterogeneity win . I say 29:56.369 --> 29:58.425 that again because I want you all to 29:58.425 --> 30:00.536 hear it because it's what I need from 30:00.536 --> 30:02.202 you . As desperate partners , 30:02.202 --> 30:04.369 heterogeneity has to win . It doesn't 30:04.380 --> 30:07.530 make any sense for something as complex 30:07.540 --> 30:10.729 as A I and as distributed as the US 30:10.739 --> 30:12.750 department of defense to say that 30:12.760 --> 30:15.670 there's one pheno solution that makes 30:15.680 --> 30:19.160 zero sense . Plus if only one V 30:19.390 --> 30:21.530 wins , there's a lot of lawsuits 30:24.439 --> 30:26.050 that was on the record for . 30:28.250 --> 30:31.709 So Joe all joking aside , I don't even 30:31.719 --> 30:33.830 care about that aspect of it . What I 30:33.830 --> 30:35.941 care about is that the right solution 30:35.941 --> 30:38.630 for the right job in the right place . 30:38.640 --> 30:40.910 Is what we get to build . So we wanna 30:40.920 --> 30:43.750 build this A I scouting in such a way 30:44.109 --> 30:46.619 that vendors can show up to play , 30:46.810 --> 30:50.050 understanding the rules and that uh 30:50.060 --> 30:52.116 model builders within the department 30:52.116 --> 30:54.282 can come to us and say I have to build 30:54.282 --> 30:56.393 the following kind of models . And we 30:56.393 --> 30:58.560 have the playbook which includes the , 30:58.560 --> 31:00.727 the betters who know how to follow the 31:00.727 --> 31:02.893 rules , who abided by the right rule . 31:03.239 --> 31:05.461 And we're working very hard on this and 31:05.461 --> 31:07.683 you're gonna see a lot of this . Please 31:07.683 --> 31:09.795 pay attention to this as you're going 31:09.795 --> 31:11.961 around the symposium these next couple 31:11.961 --> 31:14.660 of days , everything I'm saying , let 31:14.670 --> 31:16.449 me make sure I get that right . 31:18.680 --> 31:20.847 I'm pretty sure most things I'm saying 31:20.847 --> 31:24.239 now are well under way . Actually , 31:24.250 --> 31:26.417 everything saying now is going the way 31:26.969 --> 31:29.080 and over the next year we're gonna do 31:29.080 --> 31:31.430 even more . But the point of this 31:31.439 --> 31:34.199 symposium , both for government and for 31:34.209 --> 31:36.376 industries to let you know where we're 31:36.376 --> 31:39.280 at . We were too quiet . We should have 31:39.290 --> 31:41.512 let you know a little bit more loudly . 31:42.170 --> 31:44.337 We're gonna fix that going forward and 31:44.337 --> 31:46.559 we're gonna start it with this simple . 31:46.559 --> 31:48.781 These are our goals . The things that I 31:48.781 --> 31:50.892 said are our goals . We need to build 31:50.892 --> 31:53.114 up this hierarchy of needs . We need to 31:53.114 --> 31:55.281 build a data mesh , we need to build A 31:55.281 --> 31:57.430 I scaping , we need to build um 31:57.439 --> 32:01.430 contracting and policy . That's agile , 32:01.439 --> 32:04.920 responsive and fast . That cost by . 32:06.459 --> 32:08.570 All right , Bonnie runs trade winds . 32:08.570 --> 32:10.792 If you don't know about trade winds get 32:10.792 --> 32:13.260 on trade winds . Yeah , there'll be 32:13.270 --> 32:15.492 plenty of people dying to tell you that 32:15.492 --> 32:17.492 there have been companies who got a 32:17.492 --> 32:20.719 contract in , in 13 days , 13 days , 32:22.050 --> 32:24.510 trade links get on trade . It's also 32:24.520 --> 32:28.250 try A I but we 32:28.260 --> 32:30.204 can't do this without you . All of 32:30.204 --> 32:32.427 these components that we're envisioning 32:32.427 --> 32:34.949 are going to be collections of 32:34.959 --> 32:37.229 industrial solutions with the right 32:37.239 --> 32:40.130 contracting . There's already been rfis 32:40.140 --> 32:42.550 for data mesh , respond to them , there 32:42.560 --> 32:45.030 will be RMIS for model monitoring , 32:45.219 --> 32:48.140 respond to them . We can't do this 32:48.150 --> 32:51.209 without you nor do we want to . I like 32:51.219 --> 32:55.130 the , the uh uh I love everybody 32:55.140 --> 32:57.790 whose belief is to save taxpayer money . 32:57.880 --> 32:59.936 I think that's a great thing . I'm a 32:59.936 --> 33:02.569 taxpayer . Another . Sure God is the 33:02.579 --> 33:05.359 right motion . I would much prefer to 33:05.369 --> 33:07.719 have the experts building what we need 33:07.729 --> 33:11.459 built and add the fast way . So 33:11.770 --> 33:13.770 that's what we're working towards . 33:13.839 --> 33:16.020 That's how you can see over the next 33:16.829 --> 33:20.589 day and a half , 2.5 days . Uh And 33:20.599 --> 33:23.160 please over the next year , over 2024 33:23.170 --> 33:25.939 pay attention to request that we have 33:25.949 --> 33:27.838 it . Oh , by the way , every talk 33:27.838 --> 33:30.329 should have a contact email and I'm not 33:30.339 --> 33:33.699 giving you my , but every talk should 33:33.709 --> 33:35.949 have a contact email and it should be 33:35.959 --> 33:38.126 up on the slide and they're done . And 33:38.126 --> 33:40.348 if it doesn't go over and ask for their 33:40.348 --> 33:42.459 contact information , we want this to 33:42.459 --> 33:44.348 be the beginning of a very strong 33:44.348 --> 33:46.126 interaction with our government 33:46.126 --> 33:48.459 partners and with our industry partners . 33:48.459 --> 33:50.459 We're very excited about what we've 33:50.459 --> 33:52.515 done over the last year . It's , I'm 33:52.515 --> 33:54.570 shocked by how much great work we've 33:54.570 --> 33:56.681 gotten done in the year and excited . 33:56.681 --> 33:58.848 Shocked . I'm excited . Not like I got 33:58.848 --> 34:01.070 you guys . I'm , I'm very excited about 34:01.070 --> 34:03.292 all the way to what we've got done over 34:03.292 --> 34:05.459 the last year . And I'm really excited 34:05.459 --> 34:07.459 about what we get done in 2024 . So 34:07.459 --> 34:09.600 please ask questions , push hard 34:09.610 --> 34:12.260 challenge us . Tell us we're crazy . 34:12.270 --> 34:14.326 None of us will be offended . I told 34:14.326 --> 34:16.548 you our view of winning is that you all 34:16.548 --> 34:18.770 think it's your idea . That's fine with 34:18.770 --> 34:21.409 me . OK ? And um I think I have a slide , 34:21.419 --> 34:23.629 a final slide that tells you the kind 34:23.639 --> 34:25.639 of things you can look for over the 34:25.639 --> 34:27.879 next year . We're gonna be publishing 34:27.889 --> 34:30.719 our API S uh for government data and 34:30.729 --> 34:32.896 our Federated data , data catalogs and 34:32.896 --> 34:35.679 developers . But those folks have to 34:35.689 --> 34:38.479 agree to share their data products in 34:38.489 --> 34:41.550 that time . So that's gonna be breaking . 34:41.560 --> 34:43.616 That's gonna be changing the rules a 34:43.616 --> 34:46.979 little bit hoarding your API S . So I 34:46.989 --> 34:49.045 understand the contracts now say the 34:49.045 --> 34:51.267 data officially belongs to us , but the 34:51.267 --> 34:53.433 data that officially belongs to us and 34:53.433 --> 34:56.770 the Terabyte Json is not useful . OK ? 34:57.629 --> 34:59.851 The day that officially belonging to us 35:00.239 --> 35:03.959 successful via an API Don't worry , I 35:03.969 --> 35:06.302 want you to make money for these things . 35:06.302 --> 35:08.302 We can talk about what the contract 35:08.302 --> 35:10.358 looks like . And if , if you have ID 35:10.358 --> 35:12.580 behind that , let's talk about the cost 35:12.580 --> 35:14.636 of being able to use that . IP iii I 35:14.636 --> 35:17.489 want you to think about not um is this 35:17.500 --> 35:19.667 gonna make me lose money ? But how can 35:19.667 --> 35:21.778 I switch the mentality and still make 35:21.778 --> 35:23.722 money ? That's the conversation we 35:23.722 --> 35:25.833 wanna have . But we really do need to 35:25.833 --> 35:27.889 be having folks who come to play and 35:27.889 --> 35:29.944 who have put together data the right 35:29.944 --> 35:32.167 way , make that data accessible through 35:32.167 --> 35:35.250 the cat . OK ? We'll be creating a hub 35:35.260 --> 35:37.260 of label data and the collection of 35:37.260 --> 35:39.371 test and evaluation tools . Oh By the 35:39.371 --> 35:41.371 way , our TV tools are just kicking 35:41.371 --> 35:43.427 butt . Uh That , that team is really 35:43.427 --> 35:45.593 remarkable . If you haven't seen those 35:45.593 --> 35:47.427 tool kits look them up . I think 35:47.427 --> 35:47.379 they're about available available in a 35:47.389 --> 35:49.611 i.mil . Oh By the way , at some point , 35:49.611 --> 35:51.833 we'll switch A I dot mail to CD A O dot 35:51.833 --> 35:54.056 ne . Uh things move slowly . I've tried 35:54.056 --> 35:57.850 that for a year now . Uh uh 35:57.860 --> 36:00.090 And so develop so developers to learn 36:00.100 --> 36:02.679 what does it mean to be a responsible 36:03.060 --> 36:05.227 uh to build a responsible A IR because 36:05.227 --> 36:07.729 we have clear views about that . Uh And 36:07.739 --> 36:09.739 we have a replicator up there as an 36:09.739 --> 36:11.683 example because there's an example 36:11.683 --> 36:14.179 where we are providing a hub of label 36:14.189 --> 36:17.020 data for the replicator participants to 36:17.030 --> 36:19.389 com and fill models together on that 36:19.399 --> 36:22.639 label data . So be on the lookout and 36:22.649 --> 36:26.449 more things like that uh the 36:26.459 --> 36:28.510 guide series of experimentation has 36:28.520 --> 36:30.798 driven so much of what we learned here . 36:30.798 --> 36:32.798 So much of what we just said that I 36:32.798 --> 36:34.853 just said in the last 40 minutes has 36:34.853 --> 36:36.798 been driven by the guide series of 36:36.798 --> 36:39.879 experimentation . Um There we have 36:39.909 --> 36:41.631 builders sitting right next to 36:41.631 --> 36:44.050 operators figuring out what's the right , 36:44.060 --> 36:46.282 it's very agile figuring out what's the 36:46.282 --> 36:48.282 right flow . So you can get out the 36:48.282 --> 36:50.489 powerpoint email . So you can figure 36:50.500 --> 36:52.556 out how J two talks to J three in an 36:52.556 --> 36:54.979 efficient automated way as opposed to 36:54.989 --> 36:58.020 powerpoint indoor ski chair . Like it's 36:58.030 --> 37:00.252 remarkable to me the degree which these 37:00.252 --> 37:02.030 things still have really strong 37:02.030 --> 37:04.086 boundaries in it . Um So we're gonna 37:04.086 --> 37:06.308 continue that kind of experimentation . 37:06.308 --> 37:08.419 We're gonna drive more , be sponsored 37:08.419 --> 37:10.474 some hackathons . Uh And we have one 37:10.474 --> 37:12.641 more coming up um With the Air Force , 37:12.641 --> 37:14.752 those hackathons were good , but they 37:14.752 --> 37:16.974 weren't good enough . We have to figure 37:16.974 --> 37:19.086 out the other piece , which is how do 37:19.086 --> 37:21.719 you go from a hack together solution to 37:21.729 --> 37:23.840 a sustainable solution that has clear 37:23.840 --> 37:26.007 evaluation and TNE criteria . So hacks 37:26.280 --> 37:28.224 are great for getting , have fun . 37:28.224 --> 37:30.391 Hackathons are great to come up with a 37:30.391 --> 37:32.502 quick idea . But one that quick ideas 37:32.502 --> 37:34.836 and we saw some that might think , gosh , 37:34.836 --> 37:36.891 I wish that were deployed a deployed 37:36.891 --> 37:40.580 solution . But ok , so E 37:40.729 --> 37:43.000 five creates a solution in the 37:43.010 --> 37:45.580 hackathon . You deploy the war happens 37:46.000 --> 37:48.040 that E five goes somewhere else who 37:48.050 --> 37:50.050 fixes it in the middle of the night 37:50.050 --> 37:51.939 when it's broken , that is not an 37:51.939 --> 37:54.161 acceptable answer , right ? Half of the 37:54.161 --> 37:54.040 other solutions is not , are not 37:54.050 --> 37:55.994 acceptable answers . So we have to 37:55.994 --> 37:58.106 learn to be a like that and have them 37:58.106 --> 37:59.994 be deployed correctly . You'll be 37:59.994 --> 38:02.050 seeing more of that , those kinds of 38:02.050 --> 38:04.106 experimentation coming up . Uh We're 38:04.106 --> 38:03.790 gonna be expanding the trade mens 38:03.800 --> 38:05.744 ecosystem . Get on trades , get on 38:05.744 --> 38:07.800 trade points , get on trade points , 38:07.800 --> 38:11.040 get on trade 13 day contract . Yeah , 38:11.050 --> 38:13.580 that's it . Uh And as I already 38:13.590 --> 38:15.646 mentioned , we're gonna be coding uh 38:15.646 --> 38:18.060 positions for uh for the talent that 38:18.070 --> 38:21.600 they are uh using these new um these 38:21.610 --> 38:25.429 new dod positions and that alone is 38:25.439 --> 38:28.689 going to uh change the way commanders 38:28.699 --> 38:30.810 think about their workforces . II I , 38:30.810 --> 38:32.921 this is one thing I've learned in the 38:32.921 --> 38:32.780 government . There are some 38:32.790 --> 38:34.901 bureaucratic things that are way more 38:34.901 --> 38:36.734 powerful than technical things . 38:36.734 --> 38:38.790 Telling a man you have 10 people who 38:38.790 --> 38:40.790 are data scientists even though you 38:40.790 --> 38:42.846 didn't know it . Three years later , 38:42.846 --> 38:44.790 there'll be 10 people who are data 38:44.790 --> 38:46.957 scientists . That's why like that is a 38:46.957 --> 38:48.846 remarkable change . We're pushing 38:48.846 --> 38:50.846 really hard on that . And uh you'll 38:50.846 --> 38:52.957 hear there's gonna be a lot of talent 38:52.957 --> 38:55.179 talks and you're gonna hear a lot about 38:55.179 --> 38:54.550 how we're trying to get up to this . 38:55.580 --> 38:57.636 All right , I'm gonna stop there . I 38:57.636 --> 38:59.802 have 18 minutes left for questions and 38:59.802 --> 38:59.639 answers . Thank you for your time .