Doug Roberts: Welcome everyone
Doug Roberts: where I'm holding this presentation. because we've been doing so at various shark net institutions to welcome new potential researchers.
Doug Roberts: to
Doug Roberts: to our consortium of institutions that provide these supercomputing resources for everyone.
Doug Roberts: So our intention here is to raise awareness of the facilities that are available to you
Doug Roberts: as a Laurier researcher as they are for other universities through throughout Ontario.
Doug Roberts: So we've got a
Doug Roberts: I'm giving this as a high level overview of all the resources and capabilities that you have access to. So let's just get going on the first slide here. So we're accessing topic is accessing super computers, clouds, storage.
Doug Roberts: support, and training courses. So all these things go together.
Doug Roberts: Sort of as a unit to
Doug Roberts: to provide you this the ability to carry out advanced research computing related research.
Doug Roberts: So it's a bit of a it's a wide scope of things that need to be looked at in this presentation so that you have a good perspective on
Doug Roberts: what's available to you.
Doug Roberts: So probably wondering,
Doug Roberts: firstly, why do you need super computers? And once you've determined this, what's available to you
Doug Roberts: and how to access them.
Doug Roberts: Manage your files. run programs.
Doug Roberts: basically things that might initially plan on doing on your desktop. But pretty quickly. You'll realize
Doug Roberts: that the CPU power, the storage
Doug Roberts: that you have on your desktop is not capable
Doug Roberts: of doing the tasks that need to be done to carry out research
Doug Roberts: competitive research with against other researchers
Doug Roberts: and fulfill requirements of your program.
Doug Roberts: very well.
Doug Roberts: So this is why you need to turn to a supercomputing facility
Doug Roberts: to conduct your research.
Doug Roberts: And I'm gonna try to describe how to do that
Doug Roberts: along with what's available.
Doug Roberts: So put in bullet form. Why you need these resources.
Doug Roberts: As I mentioned, your desktop won't have enough cores or memory, most likely.
Doug Roberts: or a Gpu which is needed. A professional gpu that's capable of running for like a long period of time.
Doug Roberts: without causing any problems to the system itself. You probably need a lot of disk space
Doug Roberts: which the supercomputing facilities that we have provide. and these can vary from a small amount of disk space to a very large amount.
Doug Roberts: None of these things are like
Doug Roberts: mandatory requirements to use the facilities. So if you don't need much memory, or cores, or Gpus, or disbelace, or you just need one of them.
Doug Roberts: Then that's perfectly fine. You don't have to feel well.
Doug Roberts: I don't. I don't have all these exact needs, therefore I really don't need to go there, or they won't want me there. Something like that. Not everybody needs everything. Some people do, but not everyone does. So
Doug Roberts: I wouldn't be worried about that
Doug Roberts: more mature simulations or some types of simulations might have very high demand. So you'll need a lot of cores.
Doug Roberts: You might need to run a lot of simulations simultaneously. So
Doug Roberts: imagine you're running your program with an input file to evaluate a case. Well, you want to do 10 cases or a hundred or thousands. Some people do millions tasks at the same time on the systems.
Doug Roberts: So you could be any belonging to any part of that spectrum. You just wanna run a single job for a very long time. That's fine. Or you could wanna run
Doug Roberts: lot of jobs for a long time or a lot of jobs for a short time.
Doug Roberts: These are all sort of things you might want to do. There's web services available. There's SQL. Databases. If you need to run one. You can do that.
Doug Roberts: There's cloud facilities. We'll get a bit more into that. And, most importantly, everything here
Doug Roberts: on this menu is free.
Doug Roberts: So you don't have to worry about money, which is something that from the inception of Sharknet. And now Sharknet is part of a larger group which I'll go to in this next slide.
Doug Roberts: Canada wide. These facilities that I'm talking about are distributed across Canada. So we're in Ontario. So right middle there, you've got compute, Ontario.
Doug Roberts: We have a couple of systems that are part of Sharknet. You can see the sharknet logo that's in sort of the lower part of Ontario.
Doug Roberts: That's what Laurier belongs to other institutions in Ontario, like Western and Windsor. Hamilton. These places all belong to Sharknet, so were all sort of a community
Doug Roberts: known as sort of a consortium
Doug Roberts: called sharkness.
Doug Roberts: There's also another one Ontario, a bit of an outlier called cynet, which is specific for U of T researchers. So they have their sort of own consortium. But UT. Is quite a bit larger upon itself. So
Doug Roberts: this is how things have evolved all the other Ontario universities comprise sharp net, and then you've got
Doug Roberts: another one in Ontario.
Doug Roberts: called sign it. So outside we've got in Quebec. Very strong group support groups in Quebec.
Doug Roberts: It's known that consortium Universities there is known as Calcule Quebec. Keep going east. You've got a snack and west.
Doug Roberts: Go all the way out to BC. You've got cedar cluster. These are the cluster names I am talking about now. Cedar
Doug Roberts: Is within
Doug Roberts: the western part
Doug Roberts: out east in Quebec. You've got clusters sort of showing their Narvall and Beluga in Ontario. We've got Graham and Niagara. All this will get
Doug Roberts: you'll get more comfortable with this. When you know the names of the the compute systems themselves have names just like we do. So you become sort of familiar with it by their name.
Doug Roberts: and you can access anything across Canada cause they're all different. Whatever suits your needs. You go to the place that best matches your needs. I think if you read the abstract that I sent out, I put a little sense in there about that. So we've got
Doug Roberts: a little animation here to show of the inside of a server room in case you're wondering what they look at. Look like. Oh, okay, it's running on the other screen. I'm sorry about that
probably cannot see it.
Doug Roberts: So anyway, we got 2 shots here.
Doug Roberts: These are the back of servers just showing the cables that connect them.
Doug Roberts: essentially, that what that's what you have
Doug Roberts: in the server room
Doug Roberts: is servers that are stacked into racks, and the racks are all connected into aisles, and these things generate a lot of heat, and they're locked away in safe places because it's highly secure. And it's highly intensive to keep these things cool in a proper environment that alone involves
Doug Roberts: lot of people at the universities that
Doug Roberts: allow these clusters to be located there. So originally, Sharknet, we had clusters. When Sharknet was born. It was born at Western, very strong support there, and clusters, a server room evolved with clusters. And then there's some clusters across campus.
Doug Roberts: This spread to Hamilton. They had a cluster there. Now the primary cluster is in Waterloo, called Graham, as I mentioned.
Doug Roberts: and so you can just imagine that this room is very loud.
Doug Roberts: It can be very hot, depending which aisle you're in or cool.
Doug Roberts: And
this is where all the compute action occurs.
Doug Roberts: So here's a diagram that shows sort of how things interoperate. If you could say you've got the clusters, which are primary sort of workforce as a computing. That's what you log into and
Doug Roberts: submit jobs. Each cluster has, like thousands and thousands of servers and tens of thousands of cores and
Doug Roberts: countless amount of memory. When you add it all up.
Doug Roberts: and either clusters have storage attached to them, which is the green circle.
Doug Roberts: Then you've got cloud, which is something else. These are smaller systems, but they also have storage. This is why it's overlapping, and
Doug Roberts: the clouds can can in theory work with clusters and
Doug Roberts: everything sort of can be working together so you could be in one or the other or overlapping all of them. For sure. You're probably gonna be in the storage screen one so that's the layout. 3 primary components to think about.
Doug Roberts: So how do you get started? And this was mentioned in the abstract. So basically, you need to go sign up. And you do that at this website. We call it Ccdb
Doug Roberts: sort of the short form, and you can get it that from that Earl. your supervisor, firstly needs to get an account. He gets a username, and then you go to him and you can get his
Doug Roberts: something called a ccri.
Doug Roberts: which when you apply for account, you have to put in basically his identifier, which is that.
Doug Roberts: And then you hit, submit, and it's approved through process. And then you can simply you'll get a username and you can log in that it. It's just as easy as that.
Doug Roberts: So you can really get started today. You could even be doing while I'm talking it, probably. If your supervisor has an account already, that's that can be. Most of them do by now, but there could be pi, so faculty and attendance to this meeting.
Doug Roberts: and if they're interested in using, certainly they want to
Doug Roberts: go get an account, so their students can then get accounts. That's what I'm trying to say.
Doug Roberts: Alright. So I mentioned, there's some clusters across the country here they are, and they've all got names. So to get familiar with. Graham's at Waterloo. Oh, my apologies.
Doug Roberts: that's a problem
Doug Roberts: I can't mute that
Doug Roberts: Sorry. Just
Doug Roberts: let's just Paul, can you mute me?
Paul Preney: You have to mute yourself. Okay, whatever I'll figure it out.
Doug Roberts: Thank you
Doug Roberts: alright. Thank you. There's nothing.
Doug Roberts: So yes, Graham is at Waterloo, which is just down the street from Laurier.
Doug Roberts: They give tours once in a while. You could probably go over and see it. So here's a number after there 41,500 C. What is that? That's the number, of course. How many course does your laptop have?
Doug Roberts: Probably 4, something like that nowadays previously you only had one, but nowadays probably 4 at least.
Doug Roberts: But this thing has 41,000. Look at the other clusters. 94,000. 80,030. You get the idea. There's a lot of compute capacity more enough there to satisfy everybody's needs. That's and we're looking at. Then this is an older version of Gram relative to the other systems.
Doug Roberts: We'll be getting a newer gram pretty soon at Waterloo, and there'll be more to add to the collective. But it doesn't really matter where these resources are located. Because you can go, you you use which everyone you need to. You just connect into it and use it. But
Doug Roberts: yeah, we do sort of value Graham, a little bit more, cause we're at shark net. And that's our focus. We have a lot of Sys Admins within Sharknet, who's primary
Doug Roberts: job is to ensure that Graham runs smoothly
Doug Roberts: and meets the needs of everyone from around the country that connects into shark net. To use the sharknet systems.
Doug Roberts: we also handle like problem, the ticket. But ticket questions that come in for Graham, the staff of Sharknet, the other consortiums are like that as well. They have IT. Staff that support their clusters in their provinces.
Doug Roberts: and also support staff. Anyway, that's too much on the clusters. We've the clouds likewise have some names.
Doug Roberts: some are in common, others are not. This arbitris is a different name. It's not common cluster, and you've got east. So name but 2 names of the clouds do overlap with
Doug Roberts: cluster names coincidentally. There's different types of gpus that are on the clusters
Doug Roberts: lot. More researchers are now using incorporating Gpus to speed up their codes
Doug Roberts: and investigating how to do that. That's a little bit next level in your ability at programming and
Doug Roberts: sort of things like this. It's just a bit more complex.
Doug Roberts: but may be well worth your while to look into Gpu use if you're looking at research, and you discover that some of the research is being done with Gpus. Well, we have Gpus that are state of the art to here, and certainly you could go down that path with your research utilizing this sort of resource in addition to the cpus on the clusters.
Doug Roberts: And then there's some storage. As I mentioned. we got the
Doug Roberts: different file systems. My goal here is not to go into too much detail. Break things down.
Doug Roberts: Just say there's different types of disk storage
Doug Roberts: where you put your files that some of them perform differently.
Doug Roberts: Fast and slow. Weathers have a lot more space. Others are designed to keep your data for a very long time.
Doug Roberts: while others it'll just be gone 60 days.
You don't do anything
Doug Roberts: we have tape back up. You've probably heard of that. Tapes still do exist.
And
Doug Roberts: yeah, they're an important part of the infrastructure here. If something happens, you can get your data back on tape
Doug Roberts: pretty reliably. So that's something real peace of mind. Some. It's a real concern, you know. What could happen
Doug Roberts: to your data. What would you do if you lost everything? Well.
Doug Roberts: we're take a lot of they very seriously to support that. That doesn't happen.
Doug Roberts: Okay, so a little bit about the servers, the operating system that runs on a mix Linux base. It's not windows.
Doug Roberts: So that may not may or may not mean much to you, but
Doug Roberts: it really does. It impacts how we use the systems. So you're probably used to on windows sort of the point and click kind of thing on the super computers. You're more or less typing in commands on the command line and kind of it's like learning a new language.
You've got languages, as far as computing goes.
Doug Roberts: Probably heard of Fortran. See, they're all there. Python is very popular. Everybody's using python nowadays
Doug Roberts: seem like back in the beginning. Everybody was trying to use.
Doug Roberts: you know, see? Or Fortran to make their codes run fast. Now it seems more about
Doug Roberts: flexibility and number of free programs around that you can use and build into your
Doug Roberts: solving your problem. So Python is there for that.
Doug Roberts: And commercial
Doug Roberts: software like Matlab
Doug Roberts: to have a might.
Doug Roberts: There's other things that are open source like Julia, and to help you use the Gpus languages like Kuda. Maybe too much detail.
Doug Roberts: If you wanna be using all these cores to speed up your code, then you need to
Doug Roberts: deal with porting it to be parallel.
Doug Roberts: So you might use something like Mpi to distribute program over many servers. Open Mpa. Distributed within a server over the course in parallel that can be really efficient.
Doug Roberts: Other things to speed it up
Doug Roberts: cuda, and
Doug Roberts: everything else is porting parallel. So really depends what you wanna do. You're gonna need support yourself to probably do this. This is where the experts come in that are available within shark net and across the country to answer your questions, how to do this. And first step is a new user.
Doug Roberts: Deciding what you're gonna do is probably the most important thing you'll ever do.
Doug Roberts: As in your
Doug Roberts: HPC. Career, because if you go off in the wrong direction. It can be very time consuming to restart everything. When you realize. Oh, I should have done it this way. I'll never solve the problem this other way. I gotta go back and do it this way. It could cost you.
Doug Roberts: you know, a year easily of hard work
Doug Roberts: to restart something like that. So in the beginning take the time to look at all the options, familiarize, familiarize yourself if you decide to use the systems
and all sorts applications at the bottom there mentioned again. Python.
Doug Roberts: Julia, things
Doug Roberts: alright. So how do you connect to the systems?
Doug Roberts: Zoom, you've decided you you're gonna use Sharknet or the Alliance. All these systems we use secure protocol to connect so it can be like a command line tool on a Linux system
Doug Roberts: or a client on your windows system.
Doug Roberts: You just sort of download and click it and then specify the name of the
Doug Roberts: cluster that you connect to. And then you'll just basically, once you have your authentication, which was known as password setup. All that sorted out. You'll just log right in.
Doug Roberts: And so there's just a screenshot the lower right there, showing the display. The the mob X term
Doug Roberts: lower right corner how it might look. It's got a prompt there in yellow.
Doug Roberts: Another way to connect is over. Vmc.
Doug Roberts: Gives you a viewer. So you do. Top left. It's kind of like
Doug Roberts: analogous to obey term.
Doug Roberts: You specify the cluster here connecting to which Sharknet is something kind of unique to it.
Doug Roberts: Which we put a lot of effort in to give users the ability to run graph programs graphically. So when you connect to this system, you get your desk, you get a desktop. So
Doug Roberts: while you're looking at your desktop, this other window will pop up with a remote desktop on your desktop, and inside that you can run all your graphical applications. Really easy. This can be done in the cluster as well, but
Doug Roberts: on the gray Vdi system that it's sort of. You can just leave it sitting there and work away, and there's not a
Doug Roberts: you're you don't have to work through the schedule, or some simple, simpler aspects of it, and it gives good performance with
Doug Roberts: Gpu rendering for
Doug Roberts: rotations. Things might need to do some heavy graphs with. So
Doug Roberts: that's there. It's called Vmc. Grey. Vdi is mentioned, and
Doug Roberts: it's also really useful for setting up simulations in the native Gui environments of some really advanced softwares that are like Matlab. You want to work in the Gui, probably to work on everything right and get it set up.
Doug Roberts: and so you can do that over a remote desktop.
Doug Roberts: There's another way to run within a graphical
Doug Roberts: Workspace on the clusters. This is on the clusters. Now I'm talking about and it's over something called Jupiter Hub.
Doug Roberts: and you can.
Doug Roberts: It's pre-configured. It's got Jupiter notebooks and
Doug Roberts: Python
Doug Roberts: everything is inside of this. It's another sort of an environment to work graphically on the clusters. That you can utilize if you've heard about it, or if you hear about it again, it's there.
Doug Roberts: and then you may want to do some cloud computing. So I mention the clouds. Each of the clouds says names. We've got 4 clouds.
Doug Roberts: I can't remember the core count. So we're showing back on that slide.
Doug Roberts: let's just go back.
Doug Roberts: No, we didn't show it
Doug Roberts: so I can't tell you how many cores are in these different clouds. But there'd be a lot again. Lot of cores and you work
Doug Roberts: within openstack. You can. You're responsible for configuring everything from the operating operating system up performing updates. And you can run web servers.
Doug Roberts: All sorts of interesting things are just not in databases and what not. You can provide access to
Doug Roberts: apps and things for other researchers to use. but on highly developed projects.
Doug Roberts: what not. It's quite a bit
Doug Roberts: difference. Sort of
Doug Roberts: what's the word?
Doug Roberts: Kind of a companion to the clusters. It's not the right word, but sort of thing. Let's you do different things you can do on the cluster. When you think about it.
Doug Roberts: We can do some computing in there as well.
Doug Roberts: All right. So to trend. You'll be needing to transfer data back and forth to the systems or between systems. So you can just
Doug Roberts: some again, a little bit more detail in for new user
Doug Roberts: new users. But to help you
Doug Roberts: to be a
Doug Roberts: sort of aware that something called globus is available to you in future. When you start dealing with a lot of data. There is this sort of graphical tool that's gonna
Doug Roberts: work.
Doug Roberts: Are we efficiently sort of optimized manner to transfer the data that you've got to something called endpoints between these endpoints. It's all it's all to do. This efficiently
Doug Roberts: does require some sort of infrastructure. It's not easy to do yourself.
Doug Roberts: So that's why this is there as as a tool. So data transfer is something that impacts you in the future. Think about Lobos.
Doug Roberts: II keep talking, and the information keeps piling up. And no, I've not scratched even like fraction 1% of it. So. But we do keep everything we do. There is document here readily readily available.
Doug Roberts: These links, this link on the left, primarily now used to be on the right link, but
Doug Roberts: as sharknets now become part of the Alliance. All the other
Doug Roberts: people, like all the other shark nets of Canada. Everybody is putting all the documentation into this alliance link. So everything is there? It's open. You don't need a password to connect.
Doug Roberts: and every package has documentation, every system does everything I've talked about like clobus and Vdi
mobax term file systems, the structure. It's all here
Doug Roberts: in docs.alliance.ca. so
Doug Roberts: probably everything you need is there? And if something is not clear, then you can open a ticket and ask a question
Doug Roberts: both that
Doug Roberts: the link on the right for Sharknet is showing things like events like the new user seminar. So if you decide to get an account, then you want, first thing you're gonna want to do is attend the new user seminar and it'll go into more.
Doug Roberts: a lot more targeted topics than sort of this rambling talk. That you need to know specifics, and you can ask questions. So
Doug Roberts: find that@sharknet.ca lower left corner in the events box. And there's some news news feed in the middle as well.
Doug Roberts: That information is not available in the the left hand link you will connect.
Doug Roberts: as mentioned. Here's just the events, Link, sort of. There's a calendar with that which you will find, and every
Doug Roberts: new user seminar every Tuesday you can always go back. Revisit that
Doug Roberts: if you want for a refresher. That's not a bad idea to do, and plus you might, because it's kept up to date. If things have changed a bit since, like, let's say you saw you go to it soon, and then 2 years from now
Doug Roberts: go back and revisit that new user seminar to get the latest.
Doug Roberts: The latest high points could be really valuable use of your time.
Doug Roberts: Then there's bi-weekly seminars on different topics given by the support analyst
Doug Roberts: the Sis, Sys Admins will talk about technical topics. Just a variety of things. If you had a topic you would want
Doug Roberts: covered, you could submit it and ask us, and I think that could be.
we'd be open to
Doug Roberts: to doing topics by suggestion, I'm sure, because usually we we choose them. So force it upon you. But why not?
Doug Roberts: I'm gonna use that. Twitter
Doug Roberts: got Twitter and Youtube. Lots of stuff is on Youtube. for sharknet.
Doug Roberts: And there's your link, Youtube, dot, sharp net, dot cas check that out. We've recorded almost everything. I think.
Doug Roberts: at this point and put it up there. It's
Doug Roberts: there's no no hesitation to record things and put it up so that it's not forgotten, and
Doug Roberts: the effort people put into that to present that information isn't lost with a one time sort of thing. So you can go back and you watch those, as you know, Youtube.
Doug Roberts: more on this where to get help.
Doug Roberts: So in the Alliance link that I so I gave showed you 2 slides ago. There's a support. If you look down the left menu.
Doug Roberts: it'll show you how to request for support. And basically you're going to send an email to support@techalliance.ca.
Doug Roberts: you could contact. Try to reach people at the sites
Doug Roberts: by finding the analysts. Those that are working locally. Anyway.
Doug Roberts: yeah, you can come in and talk to us, or you want to be sure we could do it virtually. Might be user easier
Doug Roberts: for people nowadays.
Doug Roberts: make some sort of arrangement. Basically, contact is, if you think you need in person support.
Doug Roberts: That's why we have a staff at each. Most of the universities within shark net
Doug Roberts: at the Ontario Universities.
Doug Roberts: At least 50% of them have staff locally. So
Doug Roberts: you could. Really, you can come in and see us
Doug Roberts: contacts. or you could contact anybody. If somebody is somebody their skills align with whatever you're doing, even if they're not at Laurier. So you want, there's a specialist at Western
Doug Roberts: or Mcmaster. or York, or where Windsor.
Doug Roberts: we will match you to that person. So this doesn't need to be a local person. If you want to talk to somebody.
Doug Roberts: Okay.
Doug Roberts: if you have projects that require more than us telling you how to fix the problem or short snippets of code, providing short snippets, saying, You know, this will fix it, or
Doug Roberts: you could do this or that. Then you want us to do the programming. There's something
Doug Roberts: available for dedicated program support, where an analyst
Doug Roberts: dedicates part of
Doug Roberts: part of his day. For a period of time
Doug Roberts: 4 months is suggested. There could be couple of days a week that he worked specifically on your problem to help move that thing forward, maybe training. At the same time, you're learning
Doug Roberts: how to handle it so you can then take it over
at some point.
Doug Roberts: So there's there's not a lot like, not everybody can do this. So
Doug Roberts: the applications are reviewed and selected and then assigned, based on certain conditions being met and the best usage of the people that we have for those.
Doug Roberts: Yeah. So
Doug Roberts: there's
Doug Roberts: never ending amount of training going on and workshops. So you can find those. As I mentioned on the sharknet site. Watch your email.
Doug Roberts: Never like, I'm sure, can. It sends email. Don't just
don't. Please don't put it into the Europe.
Doug Roberts: I don't spam folder something like, read everything cause events are always happening, and you don't want to miss them.
Doug Roberts: So
Doug Roberts: that that really is, ends the resources section.
Doug Roberts: Now, something to understand. How do you access? How do you? How does your pi
Doug Roberts: get access to the resources.
Doug Roberts: And so
Doug Roberts: pis need to apply if they know that they're gonna
Doug Roberts: be commencing on a large amount of calculations that need to be done in a certain amount of time.
Doug Roberts: Then we need to know about that in detail. They submit an application called a rack.
Doug Roberts: explaining the exact amount of storage, or cores, or servers, or computers, whatever it is that they need to get it done.
Doug Roberts: And then these applicate these applications are processed to prove and allocated onto different systems. There's a whole process for this and the Pi's need to.
Doug Roberts: They need to be on top of it essentially, and oversee their entire work group. so that this gets done
Doug Roberts: in according to the application form. Everything needs to be filled in for all the the groups all the research projects that are to be undertaken, and it's all broken down
Doug Roberts: and whatnot
Doug Roberts: so that is called
Doug Roberts: can see it rack. Just think, rack. And if you Google, the rack in the the Alliance, Wiki Link. or even just Google it with alliance, we'll get the Alliance website which gives a lot of information about rack.
Doug Roberts: So or you can email us at help. And
Doug Roberts: we've already had sessions explaining to researchers.
Doug Roberts: So the
Doug Roberts: the rack period is lasts for a year. So you apply in the fall
Doug Roberts: for your rack.
Doug Roberts: That will be for the next year, basically, that'll take place.
Doug Roberts: So by January you'll get your award, and you can start your computing based on what you were given, since quite often the requests exceed the resources available. Some of them, ha! Unfortunately are reduced.
Doug Roberts: so your expectations might not be 100% met, but
Doug Roberts: just a little bit taken off from those where it can be done so shouldn't be minimal pain in that regard.
Doug Roberts: Right? So what every graduate student should know.
Doug Roberts: Is our next topic.
Doug Roberts: Yeah.
Pawel Pomorski: thanks, gentlemen, second
Pawel Pomorski: quick quick point that the Iraq competition, the deadline is November seventh.
Pawel Pomorski: Hello, this year. You should apply immediately.
Doug Roberts: Yeah. So I mentioned, they've had some presentations about it already, and that's because the closing dates coming November the seventh, seventh, when you should have yours submitted so potentially, you could still some submit one now and get through it. But you're gonna have to
Doug Roberts: work in a dedicated manner to that to get it in. And if there's questions we can be contacted.ca.
Doug Roberts: alright. So what graduate students should know.
Doug Roberts: or rather, what? What do they do?
Doug Roberts: What don't they do?
Doug Roberts: We can. Almost everything is done and computing area by everybody. And it's not just hardcore scientists. It's
Doug Roberts: every virtually from every department within the universities. Now, as you know.
Doug Roberts: utilizing computers, especially through a I. Things are becoming more accessible, like explosion of accessibility to advanced information. But the usual.
Doug Roberts: Some of the common areas are there in bullet points.
Doug Roberts: economics, finance.
Doug Roberts: interesting
Doug Roberts: stuff. In the scientific field mentioned middle Boat. There are couple of simulations between different solvers, sort of interesting and a bit new.
Doug Roberts: Traditional large scale. Astro astrophysics, stuff
Doug Roberts: weather things back here.
Doug Roberts: A lot of this nonlinear stuff requires a large, large, large amount of computing resources.
and there's some meshing of some domains shown.
Doug Roberts: So I think I've covered everything. What you should know hopefully, from what I've said is now being summarized again.
Doug Roberts: You've got thousands of computers, cores, Gpus.
Doug Roberts: everything to help you get the work done faster if you do it properly. This important point, if you just, you know it's sort of like garbage in garbage out. If you don't take the time to do something
Doug Roberts: properly, it could take a long time to finish. If you take a little bit of extra effort and time to use proper libraries, or better coding things like that. It could run 10 times faster, 100 times faster. So, sitting around for days. Wait for something. You could be have it done in a minute. So that's sort of the objective is to do the work faster.
Doug Roberts: and we need to be.
Doug Roberts: It's, you know, the the equipment can only go so far and speeding things up. It's the
Doug Roberts: human side of it is still critically important, and that's where all the training comes in doing things, and very plan, manner per se.
Doug Roberts: So the computers around Linux. I mentioned that not windows that could have some implications for software that you're used to using right now. So we wanna be aware of that.
Doug Roberts: it if you want to. If there's something in windows. Essentially, there's something equivalent in Linux that's available
Doug Roberts: that could be could be used. And we can help make those sort of. If you tell us what you're using, we can come back and say, Well, you know, because we we hear this all the time you can use this within the Linux on the systems. And it'll do basically the same thing, or hopefully, a lot better.
Doug Roberts: Alright. So the education extremely important. You want to look at all these things mention algorithms, miracle methods
Doug Roberts: some are mature and ready to use. And you wanna make sure that you don't repeat reinvent the wheel. because. super computing has been around a long time now, and
Doug Roberts: almost everything's been done. And there's probably special tool. If you're thinking about doing something.
Doug Roberts: look for that tool that's already been. It's already out there and available to you, even if it's not on the system which it probably is, we can put it on there for you that that could also mean new techniques for using these these packages.
Doug Roberts: Basically on a Linux system. There's commands command line. And you can Google it? You can attend our training seminars, the new user one
Doug Roberts: and get yourself comfortable with using Linux, probably in a couple of days. And
Doug Roberts: if you're on windows you can. So you can simulate using Linux through the windows subsystem for Linux. and we've got a great course for that. We've got some great courses and
Doug Roberts: if you attend that, you can learn exactly how to
Doug Roberts: turn your windows machine into sort of a Linux, a Linux tool for your research.
Doug Roberts: It's very pretty nice. okay. A lot of these things I've mentioned already.
Highly tuned libraries is here
Doug Roberts: down towards the bottom. Didn't mention yet, but for writing code there's
Doug Roberts: codes
Doug Roberts: that help debug your code. So this code or application, you run it and you tell it to run your code, and it comes back and tells you what's wrong with your code, how to speed it up or optimize it where the bugs are, and some of these are called, here's a free one. Gdb.
Doug Roberts: there's something very expensive which we have very
Doug Roberts: sort of. The elite in the world of debugging is Ddt. If you can get your hands on that and learn how to use that. You can, you know, run your program in real time and watch it, work through everything in your code, and you can do that in python almost any language is supported. It's all in Gui everything. So
Doug Roberts: it'll help you get from your code which you put down into something that's working really efficiently and not stuck somewhere on some bug which can be very time consuming to find.
Doug Roberts: That's why that tool is there?
Doug Roberts: Alright, yes, machine learning is ported things like tensorflow and pi torch. This is all now been around for a while and very mature, and
Doug Roberts: we have a lot of support for this stuff for people that know exactly how to implement it and stuff. So don't hesitate to ask.
Doug Roberts: You're wondering if we do that. Yes, we totally do this, and obviously we have enough compute capacity to do it seriously. Visualization I mentioned, there's some degree Vdi machine. You can do it on the clusters. But ultimately the applications you run, and a popular one for us is called Pair View. Something else called visit. Those are both open source. You can run it at home, too.
Doug Roberts: So
Doug Roberts: you're not just sort of
Doug Roberts: stuck using it on our systems. It's portable per se. Ultimately, you want all your research to be portable from what you do on our systems, that you can take it with you wherever you go.
Doug Roberts: That's always part.
Doug Roberts: We're we're within that ecosystem of the global Hpc community.
Doug Roberts: And so that's sort of in mind with all the design that we have here.
Doug Roberts: there's a link to take note of. Wanna jot it down training.sharknet.ca.
Doug Roberts: it's probably where you're at now as a new user. So important.
Doug Roberts: Alright! What are the training courses more on training
Doug Roberts: all all these topics which I talked about? Everything has a training course, and even in parallel.
Doug Roberts: So you don't have to. That's let let
Doug Roberts: our specialists give you the information in the most efficient way, so you can learn it really, quickly
Doug Roberts: and hopefully, properly.
Doug Roberts: and you can work through these bullets. these initial bullets
Doug Roberts: down in.
Doug Roberts: then go into whichever is specific to your research near the bottom.
Doug Roberts: So again, if you want the training courses. You need to have a supervisor. You need an alliance with an alliance account.
Doug Roberts: so they're not open it. Then this. There's a reason for this we need to. We report all the amount of support that we deliver to
Doug Roberts: the funding agencies, and
Doug Roberts: therefore they're not like we can track it. If you have an alliance, count and whatnot so. and presumably a lot of what we're training you for is
Doug Roberts: to help you use on our systems, too. So it kind of makes sense. We're not training you for
Doug Roberts: how to write a course.
Doug Roberts: you're looking for course assistance we still have to. You can still attend. But you've got to go through the formality of. we need to know that your alliance user, basically. So
Doug Roberts: live classes. Now, it's becoming self driven, driven a lot of things before we used to go around the universities and present for 2 or 3 days all the courses in parallel streams. Yeah, pick and choose which one you'd want can look at others because others are running. But now it's with, you know, the remote developments and everything. It's affected us a lot. And now.
Doug Roberts: looking at giving courses are now more available offline, so you can use it when you're at your convenience and whatnot. everything's offline. You can do all that. You don't need to be in attendance somewhere, traveling some other place. To do it so ultimately makes it easier to learn
Doug Roberts: more convenient.
Doug Roberts: Training courses.
Doug Roberts: Alright. So yes, it's remote. It's over zoom, just like we're doing now, so it wouldn't be much different. And
Doug Roberts: one option or self-paced. So basically, we've got both of them.
Doug Roberts: Sure, I think I've I've hit the end here
Doug Roberts: all right. So that's it. We've made it to the end hopefully a convinced
Doug Roberts: whoever's in attendance here we got 9. Some of them are staff.
Doug Roberts: Hopefully. Some lawyer users will
Doug Roberts: be motivated to
Doug Roberts: get an account on the Alliance after this. Get through, Supervisor, to get an account
dumb.
Doug Roberts: Yeah.
Doug Roberts: thanks for attending. Yeah, we'll open up for questions. I'll stop the recording, and we'll open up the questions, and you can unmute yourself or ask
Paul Preney: to ask question, or put it, put it in the chat.